Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Add docs for runtime fields #62653

Merged

Conversation

lockewritesdocs
Copy link
Contributor

@lockewritesdocs lockewritesdocs commented Sep 18, 2020

Adding a new documentation page for runtime fields. Also adding a short description of runtime fields on the field data type page.

Relates to #59332

@lockewritesdocs lockewritesdocs self-assigned this Sep 18, 2020
Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a start! I left a few notes about stuff that I think is worth changing.

I'm also wondering "where do we go from here?" This is a fairly large new feature and a fundamental change to how we can operate. I think it is fine not to dive too too deep into it all because we've not yet built some of the key features of runtime fields like grok, but I think we should do something more than this. I just don't really know what.

docs/reference/mapping/types.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
Copy link
Contributor

@jrodewig jrodewig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@javanna asked me to take a quick look. It's a good start, but I think it can be improved by taking a step back from the mechanics of how runtime fields work. Focus on the problems runtime fields solve and why a user might want to use it.

Some other things it'd be great to see:

  • Additional guidance on how runtime fields are set up. How should users disable indexing? Are there fields where they shouldn't? Where should I store my doc values before the runtime field exists? Should I let dynamic mapping do it's thing?
  • A couple of in-depth examples that realistically show how a user would use a runtime field. Start with the raw data and end with a search on the runtime field. It'd be really cool if one of those showed how a runtime field could replace an ingest processor or a multi-field.

docs/reference/mapping/types.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
@lockewritesdocs
Copy link
Contributor Author

@elasticmachine update branch

@lockewritesdocs
Copy link
Contributor Author

@elasticmachine update branch

Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some comments and questions

docs/reference/mapping/types.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/types/runtime.asciidoc Outdated Show resolved Hide resolved
@lockewritesdocs
Copy link
Contributor Author

@elasticmachine update branch

Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a couple of minor comments, looks good otherwise!

docs/reference/mapping.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/runtime.asciidoc Outdated Show resolved Hide resolved
docs/reference/mapping/runtime.asciidoc Show resolved Hide resolved
Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM besides the last minors I left, also I think that we should have more examples with objects so that defining runtime fields as part of objects becomes more natural and we don't need to explicitly call it out. If we do leave a mention of it, be sure not to mix it up with shadowing which we document specifically.

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some small things. I'll take a more complete look in another few minutes.

docs/reference/mapping/runtime.asciidoc Show resolved Hide resolved
docs/reference/mapping/runtime.asciidoc Outdated Show resolved Hide resolved
@lockewritesdocs lockewritesdocs merged commit bce1081 into elastic:master Dec 9, 2020
@lockewritesdocs lockewritesdocs deleted the docs__add-runtime-fields branch December 9, 2020 22:55
@lockewritesdocs lockewritesdocs added >docs General docs changes v8.0.0 labels Dec 9, 2020
@elasticmachine elasticmachine added the Team:Docs Meta label for docs team label Dec 9, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@lockewritesdocs lockewritesdocs added the :Search Foundations/Mapping Index mappings, including merging and defining field types label Dec 9, 2020
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Dec 9, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@lockewritesdocs lockewritesdocs restored the docs__add-runtime-fields branch December 10, 2020 00:19
@lockewritesdocs lockewritesdocs deleted the docs__add-runtime-fields branch December 10, 2020 00:20
lockewritesdocs pushed a commit that referenced this pull request Dec 10, 2020
* [DOCS] Add docs for runtime fields (#62653)

* First steps in docs for runtime fields.

* Adding new page for runtime fields.

* Adding page for runtime fields.

* Adding more to the runtime fields topic.

* Adding parameters and retrieval options for runtime fields.

* Adding TESTSETUP for index creation.

* Incorporating review feedback.

* Incorporating reviewer feedback.

* Adding examples for runtime fields.

* Adding more context and simplifying the example.

* Changing timestamp to @timestamp throughout.

* Removing duplicate @timestamp field.

* Expanding example to hopefully fix CI builds.

* Adding skip test for result.

* Adding missing callout.

* Adding TESTRESPONSEs, which are currently broken.

* Fixing TESTRESPONSEs.

* Incorporating review feedback.

* Several clarifications, better test cases, and other changes.

* Adding missing callout in example.

* Adding substitutions to TESTRESPONSE for shorter results shown.

* Shuffling some information and adding link to script-fields.

* Fixing typo.

* Updates for API redesign -- will break builds.

* Updating examples and including info about overriding fields.

* Updating examples.

* Adding info for using runtime fields in the search request.

* Adding that queries against runtime fields are expensive.

* Incorporating feedback from reviewers.

* Minor changes from reviews.

* Adding alias for test case.

* Adding aliases to PUT example.

* Fixing test cases, for real this time.

* Updating use cases and introducing overlay throughout.

* Edits, adding 'shadowing', and explaining shadowing better.

* Streamlining tests and other changes.

* Fix formatting in example for test.

* Apply suggestions from code review

* Incorporating reviewer feedback 7 Dec

* Shifting structure of mapping page to fix cross links.

* Revisions for shadowing, overview, and other sections.

* Removing dot notation section and incorporating review changes.

* Adding updated example for shadowing.

* Streamlining shadowing example and TESTRESPONSEs.

* Adding type field in responses for 7.x to fix test cases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>docs General docs changes :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Docs Meta label for docs team Team:Search Meta label for search team v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants