Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ML] Add new feature_processing field to Data frame analytics config #59327

Closed
benwtrent opened this issue Jul 9, 2020 · 1 comment · Fixed by #60528
Closed

[ML] Add new feature_processing field to Data frame analytics config #59327

benwtrent opened this issue Jul 9, 2020 · 1 comment · Fixed by #60528
Labels
>enhancement :ml Machine learning

Comments

@benwtrent
Copy link
Member

benwtrent commented Jul 9, 2020 •
edited
Loading

Data frame analytics jobs should handle a new configuration field called feature_processing.

This field will be an array of PreProcessor objects.

Requirements:

  • feature_processing array gets serialized as a parameter to the analysis native process
  • include/exclude works with custom fields.
    • excluded fields can STILL be used in a feature_processing processor.
    • a feature_processing processed field cannot be excluded.
  • Before data is extracted and sent to the native process, the data needs to be pushed through the feature_processing processors
@benwtrent benwtrent added >enhancement :ml Machine learning labels Jul 9, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (:ml)

@benwtrent benwtrent mentioned this issue Jul 31, 2020
Merged
benwtrent added a commit that referenced this issue Aug 14, 2020
@benwtrent
[ML] adds new feature_processors field for data frame analytics (#60528)
de3107a 
feature_processors allow users to create custom features from
individual document fields.

These `feature_processors` are the same object as the trained model's pre_processors. 

They are passed to the native process and the native process then appends them to the
pre_processor array in the inference model.

closes #59327
benwtrent added a commit to benwtrent/elasticsearch that referenced this issue Aug 14, 2020
@benwtrent
[ML] adds new feature_processors field for data frame analytics (elas…
c530280 
…tic#60528)

feature_processors allow users to create custom features from
individual document fields.

These `feature_processors` are the same object as the trained model's pre_processors.

They are passed to the native process and the native process then appends them to the
pre_processor array in the inference model.

closes elastic#59327
benwtrent added a commit that referenced this issue Aug 14, 2020
@benwtrent
[ML] adds new feature_processors field for data frame analytics (#60528…
8f30228 
…) (#61148)

feature_processors allow users to create custom features from
individual document fields.

These `feature_processors` are the same object as the trained model's pre_processors.

They are passed to the native process and the native process then appends them to the
pre_processor array in the inference model.

closes #59327
@Mpdreamz Mpdreamz mentioned this issue Nov 16, 2020
Closed
61 tasks
@stevejgordon stevejgordon mentioned this issue Dec 17, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>enhancement :ml Machine learning
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[ML] adds new feature_processors field for data frame analytics benwtrent/elasticsearch
2 participants
@benwtrent @elasticmachine