Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data Prepper documentation updates: autogeneration campaign #7707

Merged
merged 24 commits into from
Jul 19, 2024
Merged

Data Prepper documentation updates: autogeneration campaign #7707

merged 24 commits into from
Jul 19, 2024

Conversation

dlvenable
Copy link
Member

Description

This is part of a campaign we have to move toward auto-generated documentation in Data Prepper.

This PR updates a few processors.

@chenqi0805

Issues Resolved

N/A

Version

N/A

Frontend features

N/A

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dlvenable
Updates Data Prepper documentation with some missing fields. Adds sup…
92253c0 
…port for autogeneration of processors by naming to match the processor and including the autogenerated comment.

Signed-off-by: David Venable <dlv@amazon.com>
chenqi0805
chenqi0805 reviewed Jul 15, 2024
View reviewed changes
_data-prepper/pipelines/configuration/processors/parse_json.md Outdated
@@ -15,11 +15,21 @@ The `parse_json` processor parses JSON data for an event, including any nested f

You can configure the `parse_json` processor with the following options.

<!--
This table is autogenerated. Do not edit it.
- name: obfuscate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Miscopy

_data-prepper/pipelines/configuration/processors/parse_json.md Outdated
<!--
This table is autogenerated. Do not edit it.
- name: obfuscate
- pluginType: parse_json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

processor

@dlvenable
Corrections from PR.
207d783 
Signed-off-by: David Venable <dlv@amazon.com>
@dlvenable
Copy link
Member Author

@chenqi0805 , Thanks for catching those errors. I pushed a new change.

@vagimeli
Update _data-prepper/pipelines/configuration/processors/convert_entry…
5270f9a 
…_type.md

Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/convert_entry…
b7be720 
…_type.md

Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/parse_ion.md
83b762d 
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/convert_entry…
a303b27 
…_type.md

Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/convert_entry…
13ca7ca 
…_type.md

Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/parse_ion.md
b42adf7 
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/parse_ion.md
10ce5cd 
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/parse_json.md
53e6bf3 
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/parse_json.md
dcb06b4 
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/parse_ion.md
c6a791e 
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
@vagimeli
Update _data-prepper/pipelines/configuration/processors/convert_entry…
fc90c15 
…_type.md

Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
vagimeli
vagimeli reviewed Jul 16, 2024
View reviewed changes
_data-prepper/pipelines/configuration/processors/convert_entry_type.md Outdated
| `keys`| Yes | Keys whose value needs to be converted to a different type. |
| `type` | No | Target type for the key-value pair. Possible values are `integer`, `long`, `double`, `big_decimal`, `string`, and `boolean`. Default value is `integer`. |
| `null_values` | No | String representation of what constitutes a `null` value. If the field value equals one of these strings, then the value is considered `null` and is converted to `null`. |
| `scale` | No | Modifies the scale when used to convert to a `big_decimal`. The default value is `0`. |
Copy link
Collaborator

@vagimeli vagimeli Jul 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dlvenable @chenqi0805 Is this saying that scale is only modified when the option is converted to big_decimal?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed a new commit that attempts to correct the wording.

We are modifying the scale of the big_decimal value itself. This field is only applicable for big_decimal.

@dlvenable
Corrections to wording from PR review.
4d55b4c 
Signed-off-by: David Venable <dlv@amazon.com>
vagimeli
vagimeli approved these changes Jul 16, 2024
View reviewed changes
Copy link
Collaborator

@vagimeli vagimeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vagimeli
Copy link
Collaborator

vagimeli commented Jul 17, 2024 •
edited
Loading

@hdhalter This PR is blocking from me merging it b/c build is failing (see image below). The link isn't broken (the hyperlink is correct https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/processors/parse-json/). A similar error for another doc link is happening in #7708 . Can I bypass the build requirement so that I can merge?
Screenshot 2024-07-17 at 11 29 32 AM

This was referenced Jul 17, 2024
Merged
Merged
@dlvenable dlvenable merged commit 27c4122 into opensearch-project:main Jul 19, 2024
5 checks passed
@dlvenable
Copy link
Member Author

@hdhalter This PR is blocking from me merging it b/c build is failing (see image below). The link isn't broken (the hyperlink is correct https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/processors/parse-json/). A similar error for another doc link is happening in #7708 . Can I bypass the build requirement so that I can merge? Screenshot 2024-07-17 at 11 29 32 AM

Merging from main has fixed this.

opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 19, 2024
@github-actions @vagimeli @hdhalter
Data Prepper documentation updates: autogeneration campaign (#7707)
92f9b30 
Updates Data Prepper documentation with some missing fields. Adds support for autogeneration of processors by naming to match the processor and including the autogenerated comment.

Signed-off-by: David Venable <dlv@amazon.com>

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
(cherry picked from commit 27c4122)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Jul 19, 2024
Merged
@hdhalter
Copy link
Contributor

@dlvenable - there was a broken link. It was parse-json.md in the file and I just fixed it to be parse_json.md.

@hdhalter
Copy link
Contributor

Thanks for merging! I added the backport label.

github-actions bot pushed a commit that referenced this pull request Jul 19, 2024
@opensearch-trigger-bot
Data Prepper documentation updates: autogeneration campaign (#7707) (#…
514fd75 
…7780)
@dlvenable dlvenable mentioned this pull request Jul 19, 2024
Closed
1 task
@dlvenable dlvenable deleted the data-prepper-autogenerate-campaign-1 branch July 19, 2024 18:55
leanneeliatra pushed a commit to leanneeliatra/opensearch-documentation-website-forl that referenced this pull request Jul 24, 2024
@dlvenable @vagimeli
@hdhalter @leanneeliatra
Data Prepper documentation updates: autogeneration campaign (opensear…
279d4e5 
…ch-project#7707)

Updates Data Prepper documentation with some missing fields. Adds support for autogeneration of processors by naming to match the processor and including the autogenerated comment.

Signed-off-by: David Venable <dlv@amazon.com>

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: leanne.laceybyrne@eliatra.com <leanne.laceybyrne@eliatra.com>
sandervandegeijn pushed a commit to sandervandegeijn/documentation-website that referenced this pull request Jul 30, 2024
@dlvenable @vagimeli
@hdhalter @sandervandegeijn
Data Prepper documentation updates: autogeneration campaign (opensear…
b57f352 
…ch-project#7707)

Updates Data Prepper documentation with some missing fields. Adds support for autogeneration of processors by naming to match the processor and including the autogenerated comment.

Signed-off-by: David Venable <dlv@amazon.com>

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: Sander van de Geijn <sandervandegeijn@icloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vagimeli vagimeli vagimeli approved these changes

@chenqi0805 chenqi0805 chenqi0805 approved these changes

@hdhalter hdhalter Awaiting requested review from hdhalter

@kolchfa-aws kolchfa-aws Awaiting requested review from kolchfa-aws kolchfa-aws is a code owner

@Naarcha-AWS Naarcha-AWS Awaiting requested review from Naarcha-AWS Naarcha-AWS is a code owner

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@natebower natebower Awaiting requested review from natebower natebower is a code owner

@stephen-crawford stephen-crawford Awaiting requested review from stephen-crawford stephen-crawford is a code owner

@epugh epugh Awaiting requested review from epugh epugh is a code owner

Assignees

@vagimeli vagimeli

Labels
backport 2.15 data-prepper
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dlvenable @vagimeli @hdhalter @chenqi0805