[ML] Add better UI support for runtime fields Transforms

[qn895]

Summary

This PR adds better support for Transform and Data Frame Analytics on the UI side. Changes include:

Transforms

• Support cloning transform job that includes runtime_mappings
• Support creating and cloning transform job where index pattern has runtime fields (can be defined via Index pattern field editor - Load and save fields #89390)
• Copy the search-time runtime_mappings over to the index pattern if user opts to create new index pattern from wizard
• Add an advanced editor so that all search-time runtime_mappings (either from index pattern or from API) will be editable
• Fix the oversized buttons in the Filter -> Range controls

Editor

Pivot

• Filter terms list the values correctly for runtime fields

Latest

Previews

Data Frame Analytics

• Support previews in Data Frame Analytics index data
• Add callout message for unsupported fields for Data Frame Analytics wizard

Todos:

• Max/min aggregation not working correctly for runtime fields [Transform] Search runtime fields are not handled properly during mapping deduction. elasticsearch#68516

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios

[walterra]

I tried to clone the job fq_runtime_mappings_transform from the "Transforms examples (7.12+)" section in the "Runtime fields investigation" document we have.

When I start cloning, the source table correctly shows the runtime fields. However, the preview doesn't return any results.

When I copy the preview configuration using the copy-to-clipboard button, it's missing the runtime fields config:

POST _transform/_preview
{
  "source": {
    "index": [
      "farequote-2019"
    ]
  },
  "pivot": {
    "group_by": {
      "airline": {
        "terms": {
          "field": "airline-lower"
        }
      }
    },
    "aggregations": {
      "responsetime.avg": {
        "avg": {
          "field": "responsetime"
        }
      },
      "responsetime.max": {
        "max": {
          "field": "responsetime"
        }
      },
      "@timestamp.value_count": {
        "value_count": {
          "field": "@timestamp"
        }
      },
      "responsetime.min": {
        "min": {
          "field": "responsetime"
        }
      }
    }
  }
}

However, I can move on in the wizard and create the job, it gets cloned including the runtime field settings successfully:

[peteharverson]

Testing using an index pattern for which I've set up two runtime fields, runtime_big and airline_lower. I only get to see one of the aggs I've set up on the runtime field, and as @walterra found, I don't see the runtime_mapping section in the 'copy to clipboard' JSON:

[qn895]

I tried to clone the job fq_runtime_mappings_transform from the "Transforms examples (7.12+)" section in the "Runtime fields investigation" document we have.

Thanks for testing. The issue with the preview as well as copy to console should be fixed in the latest changes.

Testing using an index pattern for which I've set up two runtime fields, runtime_big and airline_lower. I only get to see one of the aggs I've set up on the runtime field...

There's currently an issue with showing min/max aggregation for these runtime fields which is being tracked here elastic/elasticsearch#68516

[darnautov]

no need in this check, slice doesn't add extra values to an array

Suggested change

	unsupportedRuntimeFields:
	unsupportedRuntimeFields.length > 5
	? unsupportedRuntimeFields.slice(0, 5).join(', ')
	: unsupportedRuntimeFields.join(', '),
	unsupportedRuntimeFields: unsupportedRuntimeFields.slice(0, 5).join(', ')

[qn895]

Updated here 7b91ae0

[darnautov]

we should avoid magic numbers, worth creating a constant const maxRuntimeFieldsDisplayCount = 5; or something like this

[qn895]

Updated here 7b91ae0

[darnautov]

If I'm not mistaken, in case {extraCountMsg} is empty, the sentence ends up with 2 spaces before are not supported for analysis string. I think and {count} more message should take care of adding a space.

[qn895]

I think the extra space to extraCountMsg will actually be stripped out. For example, this is what I looks like with the spaces in extraCountMsg.

And this is what it looks like as is:

[darnautov]

If it doesn't break the layout, let's keep it as is

[peteharverson]

Can the fields in the aggregations dropdown be sorted alphabetically prior to display? Currently it looks like the runtime fields are appended to the end of the list:

[darnautov]

LGTM

[qn895]

Can the fields in the aggregations dropdown be sorted alphabetically prior to display? Currently it looks like the runtime fields are appended to the end of the list:

This is now sorted alphabetically here 7b91ae0

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: f77f6a6

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
ml	1738	1739	+1
transform	214	221	+7
total			+8

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
ml	6.3MB	6.3MB	+5.5KB
transform	925.3KB	942.1KB	+16.8KB
total			+22.3KB

History

• 💔 Build #107052 failed 04219b2
• 💚 Build #106952 succeeded 8712080
• 💔 Build #106942 failed af2e8e4
• 💚 Build #106474 succeeded 860c26f
• 💔 Build #106464 failed 9323382

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kibanamachine]

💔 Backport failed

❌ 7.x: Commit could not be cherrypicked due to conflicts

To backport manually, check out the target branch and run:
node scripts/backport --pr 90363