Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Span in PPL statsByClause could be specified after fields #2720

Merged
merged 1 commit into from
Jul 9, 2024

Conversation

LantaoJin
Copy link
Member

Description

Below PPL query fails with SyntaxCheckException

source=opensearch_dashboards_sample_data_logs | stats COUNT() as cnt by response, span(timestamp, 1d)

This PR still keep the current behavior of span as the first grouping key. But allow the span being specified after fields in statsByClause. This fixing also could help on some LLM scenarios.

Issues Resolved

#2719

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Lantao Jin <ltjin@amazon.com>
@@ -188,6 +188,7 @@ statsByClause
: BY fieldList
| BY bySpanClause
| BY bySpanClause COMMA fieldList
| BY fieldList COMMA bySpanClause
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is span expression mixed in field list also valid case?

Copy link
Member

@vamsi-amazon vamsi-amazon Jun 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the changes it looks we are missing that.

: (fieldExpression | bySpanClause) (COMMA (fieldExpression | bySpanClause))*

Something like this would actually be a good idea or define a new spanFieldList.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, if this is valid, probably we should treat span expression as regular expression and do other validation outside ANTLR. Will approve now. Please check if we need to change this in future. Thanks!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably we should treat span expression as regular expression

The reason separating Span from field list AFAIK because that grouping by more than one time-span seems meaningless. And there is one span expression in Aggregation currently:

In general, it doesn't make sense that grouping by a 10-minutes span and 5-minutes span together.
But I tried in SPL, query group by two time span could be executed:

sourcetype=access_* status=200 action=purchase | bin span=10m _time as 10m_span | bin span=5m _time as 5m_span | stats count by 10m_span, 5m_span, clientip

Yes, we could refactor in future.

@penghuo penghuo merged commit c063d5e into opensearch-project:main Jul 9, 2024
21 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 9, 2024
Signed-off-by: Lantao Jin <ltjin@amazon.com>
(cherry picked from commit c063d5e)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
penghuo added a commit that referenced this pull request Jul 10, 2024
)

(cherry picked from commit c063d5e)

Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Peng Huo <penghuo@gmail.com>
manasvinibs pushed a commit to manasvinibs/sql that referenced this pull request Aug 14, 2024
jzonthemtn pushed a commit to jzonthemtn/sql that referenced this pull request Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants