Adding mutation testing to build gradle with PiTest

[MitchellGale]

Description

Adds PiTest Gradle to build.gradle in core. This adds support for Mutation testng/Analysis to our testing ability. It should help us make more effective tests for past/future features.

Why does this help?

Mutation testing is used to find missing parts of your project not covered by existing tests. It modifies (mutates) one part of code and checks if a test is killed (fails) because of the mutation. Sometimes it will make return statements return null, other times it will switch == to !=, or < with >, or || with && and lots more mutations. More mutations can be found in the default mutations column here https://pitest.org/quickstart/mutators/. Often mutation testing provides a more clear picture of the quality of tests compared to line/branch coverage alone.

If you notice mutations that "survive" your test suite, consider adding additional tests to kill the mutations.
Short presentation PDF on mutation testing MutationTesting_Presentation.pdf.

Docs for Gradle plugin for PiTest.

Currently to be manually run, but could be added to GitHub actions later. It generates an html page indicating mutations that survived and were killed by our test suite.

Only adding support for running mutation testing in core & OpenSearch modules (for now).

Steps to run

./gradlew pitest

Report will be placed in core/build/reports/pitest/index.html.
Reports can be found in artifacts under core and opensearch respectively.

Sample Report

PiTest_Results.zip

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

CVE

hcoles/pitest#1084

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #1204 (5623c1f) into main (d03c176) will decrease coverage by 2.47%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #1204      +/-   ##
============================================
- Coverage     98.31%   95.83%   -2.48%     
- Complexity     3535     3550      +15     
============================================
  Files           342      354      +12     
  Lines          8730     9420     +690     
  Branches        554      673     +119     
============================================
+ Hits           8583     9028     +445     
- Misses          142      334     +192     
- Partials          5       58      +53     

Flag	Coverage Δ
query-workbench	62.76% <ø> (?)
sql-engine	98.32% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...rc/main/java/org/opensearch/sql/storage/Table.java	100.00% <0.00%> (ø)
...c/main/java/org/opensearch/sql/expression/DSL.java	100.00% <0.00%> (ø)
...java/org/opensearch/sql/sql/parser/AstBuilder.java	100.00% <0.00%> (ø)
...org/opensearch/sql/planner/DefaultImplementor.java	100.00% <0.00%> (ø)
...opensearch/sql/planner/logical/LogicalPlanDSL.java	100.00% <0.00%> (ø)
...pensearch/sql/sql/parser/AstExpressionBuilder.java	100.00% <0.00%> (ø)
...search/sql/planner/optimizer/pattern/Patterns.java	100.00% <0.00%> (ø)
...arch/sql/expression/datetime/DateTimeFunction.java	100.00% <0.00%> (ø)
...ch/sql/planner/logical/LogicalPlanNodeVisitor.java	100.00% <0.00%> (ø)
...ch/sql/planner/optimizer/LogicalPlanOptimizer.java	100.00% <0.00%> (ø)
... and 18 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[penghuo]

Thanks for adding this. Could you help updating https://github.com/opensearch-project/sql/blob/main/DEVELOPER_GUIDE.rst also.

[Yury-Fridlyand]

Thanks for adding this. Could you help updating https://github.com/opensearch-project/sql/blob/main/DEVELOPER_GUIDE.rst also.

Done

[Yury-Fridlyand]

Do we need to backport?