[7.5] [Discover] fix histogram min interval

[markov00]

Summary

This commit fix the missing bars on the histogram of Discover.
Discover is passing a minInterval parameter in milliseconds that is not
the the minimum interval in milliseconds between the buckets. Calendar intervals
are not fixed and this value change depending on time range and the time zone.
To avoid patching the elastic-chart and upgrading the dependency we applied a temporary patch
to the histogram code. For the feature version this patch will be removed
and a correct minInterval computation will be done directly in elastic-charts.

fix #52152

to test:
add some data that spans multiple years (possibly leap years, last one was 2016)

node scripts/makelogs.js -c 1m -d 2200/1
// or import the esArchived dataset
node scripts/es_archiver.js --es-url elastic:changeme@localhost:9275 --kibana-url http://elastic:changeme@localhost:5675 load long_window_logstash

create the index pattern for that index: logstash-* or long_window_logstash*
check the behaviour on histogram when the timerange spans several years (including leap years) with different interval configuration. The issue is visible when:

• visualizing a timerange with monthly intervals that contains data in the February bucket
  timezone: all
  time range: Nov 1, 2017 @ 01:00:00.000 - Mar 21, 2018 @ 02:00:00.000
  intervals: Monthly

• visualizing a timerange with a DST time change within the range. The DST change needs to be a positive change like when the time is moved forward (= that day last less than 24 hours) like in the following situation:
  timezone: Europe/Berlin (doesn't happen with UTC timezone)
  time range: Mar 1, 2018 @ 00:00:00.000 - May 1, 2018 @ 00:00:00.000
  intervals: all except Monthly and Yearly

• visualizing a timerange with a DST time change with auto interval on a fixed/multi days interval:
  timezone: Europe/Berlin
  time range: Last 15 years - now
  intervals: every interval smaller than month (be sure that the scaled interval is scaled to 30days)

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

• [ ] This was checked for cross-browser compatibility, including a check against IE11
• [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• [ ] Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

• [ ] This was checked for breaking API changes and was labeled appropriately
• [ ] This includes a feature addition or change that requires a release note and was labeled appropriately

[nickofthyme]

LGTM, Tested on Firefox and Chrome

[kertal]

Just tested it, and while the issue is solved in the suggested tests, there seems to be a problem with very long time ranges

[markov00]

@kertal thanks for the check let me see if this is related to this other ES bug just discovered: elastic/elasticsearch#50139

[markov00]

I can confirm that I can reproduce the @kertal reported error. It's related to the discovered bug elastic/elasticsearch#50139: when using fixed_interval the returning set of buckets are not all equally distant but they differs. a 30d fixed interval could return a 30d -1h or a 30+1h intervals.... Let me find an way to fix the Discover histogram avoiding that bug

[kertal]

Issue I've reported was saved 👍
I may have found another one, when I'm defining a very short time range for e.g. 2 sec, I'm getting an ES error

this is the request sent to ES:

{
  "version": true,
  "size": 500,
  "sort": [
    {
      "@timestamp": {
        "order": "desc",
        "unmapped_type": "boolean"
      }
    }
  ],
  "_source": {
    "excludes": []
  },
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "40ms",
        "time_zone": "US/Arizona",
        "min_doc_count": 1
      }
    }
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    },
    {
      "field": "relatedContent.article:modified_time",
      "format": "date_time"
    },
    {
      "field": "relatedContent.article:published_time",
      "format": "date_time"
    },
    {
      "field": "utc_time",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "match_all": {}
        },
        {
          "range": {
            "@timestamp": {
              "format": "strict_date_optional_time",
              "gte": "2019-10-01T03:37:57.890Z",
              "lte": "2019-10-01T03:37:59.890Z"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  },
  "highlight": {
    "pre_tags": [
      "@kibana-highlighted-field@"
    ],
    "post_tags": [
      "@/kibana-highlighted-field@"
    ],
    "fields": {
      "*": {}
    },
    "fragment_size": 2147483647
  }
}

[markov00]

@kertal that seems to an ES issue with some time_zones.
If you query an index with that timezone or with some similar one

GET logstash-*/_search
{
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "1ms",
        "time_zone": "America/Phoenix",
        "min_doc_count": 1
      }
    }
  }
}

you will get the same error. With other timezones it doesn't happen

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: db785c0

History

• 💚 Build #14993 succeeded 1506513
• 💚 Build #14816 succeeded d473816

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kertal]

Code LGTM, Issue is now solve. The latest error I've found is unrelated to this issue, but a separate issue should be opened. appreciate if someone takes care of this, since I'm OOO, but else I can do this tomorrow

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[kertal]

@kertal that seems to an ES issue with some time_zones.
If you query an index with that timezone or with some similar one

GET logstash-*/_search
{
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "1ms",
        "time_zone": "America/Phoenix",
        "min_doc_count": 1
      }
    }
  }
}

you will get the same error. With other timezones it doesn't happen

thx I've opened the following issue : elastic/elasticsearch#50265