Skip to content

Commit

Permalink
Merge branch 'main' into async-shard-fetch-psatransport
Browse files Browse the repository at this point in the history
  • Loading branch information
shiv0408 committed Feb 1, 2024
2 parents f8d8a6a + 16c5257 commit d1e90a1
Show file tree
Hide file tree
Showing 196 changed files with 1,000 additions and 349 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/create-documentation-issue.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Create Issue From File
id: create-issue
uses: peter-evans/create-issue-from-file@v4
uses: peter-evans/create-issue-from-file@v5
with:
title: Add documentation related to new feature
content-filepath: ./ci/documentation/issue.md
Expand Down
20 changes: 14 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- GHA to verify checklist items completion in PR descriptions ([#10800](https://github.com/opensearch-project/OpenSearch/pull/10800))
- Allow to pass the list settings through environment variables (like [], ["a", "b", "c"], ...) ([#10625](https://github.com/opensearch-project/OpenSearch/pull/10625))
- [Admission Control] Integrate CPU AC with ResourceUsageCollector and add CPU AC stats to nodes/stats ([#10887](https://github.com/opensearch-project/OpenSearch/pull/10887))
- [S3 Repository] Add setting to control connection count for sync client ([#12028](https://github.com/opensearch-project/OpenSearch/pull/12028))

### Dependencies
- Bump `log4j-core` from 2.18.0 to 2.19.0
Expand Down Expand Up @@ -58,7 +59,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Add task completion count in search backpressure stats API ([#10028](https://github.com/opensearch-project/OpenSearch/pull/10028/))
- Deprecate CamelCase `PathHierarchy` tokenizer name in favor to lowercase `path_hierarchy` ([#10894](https://github.com/opensearch-project/OpenSearch/pull/10894))
- Switched to more reliable OpenSearch Lucene snapshot location([#11728](https://github.com/opensearch-project/OpenSearch/pull/11728))
- Added support for Google Application Default Credentials in repository-gcs ([#8394](https://github.com/opensearch-project/OpenSearch/pull/8394))

### Deprecated

Expand All @@ -85,6 +85,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Update supported version for max_shard_size parameter in Shrink API ([#11439](https://github.com/opensearch-project/OpenSearch/pull/11439))
- Fix typo in API annotation check message ([11836](https://github.com/opensearch-project/OpenSearch/pull/11836))
- Update supported version for must_exist parameter in update aliases API ([#11872](https://github.com/opensearch-project/OpenSearch/pull/11872))
- [Bug] Check phase name before SearchRequestOperationsListener onPhaseStart ([#12035](https://github.com/opensearch-project/OpenSearch/pull/12035))

### Security

Expand Down Expand Up @@ -134,6 +135,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Support index level allocation filtering for searchable snapshot index ([#11563](https://github.com/opensearch-project/OpenSearch/pull/11563))
- Add `org.opensearch.rest.MethodHandlers` and `RestController#getAllHandlers` ([11876](https://github.com/opensearch-project/OpenSearch/pull/11876))
- New DateTime format for RFC3339 compatible date fields ([#11465](https://github.com/opensearch-project/OpenSearch/pull/11465))
- Add support for Google Application Default Credentials in repository-gcs ([#8394](https://github.com/opensearch-project/OpenSearch/pull/8394))

### Dependencies
- Bumps jetty version to 9.4.52.v20230823 to fix GMS-2023-1857 ([#9822](https://github.com/opensearch-project/OpenSearch/pull/9822))
Expand All @@ -152,12 +154,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Bump `stefanzweifel/git-auto-commit-action` from 4 to 5 ([#11171](https://github.com/opensearch-project/OpenSearch/pull/11171))
- Bump `actions/github-script` from 6 to 7 ([#11271](https://github.com/opensearch-project/OpenSearch/pull/11271))
- Bump `jackson` and `jackson_databind` from 2.15.2 to 2.16.0 ([#11273](https://github.com/opensearch-project/OpenSearch/pull/11273))
- Bump `netty` from 4.1.100.Final to 4.1.104.Final ([#11294](https://github.com/opensearch-project/OpenSearch/pull/11294), [#11775](https://github.com/opensearch-project/OpenSearch/pull/11775))
- Bump `netty` from 4.1.100.Final to 4.1.106.Final ([#11294](https://github.com/opensearch-project/OpenSearch/pull/11294), [#11775](https://github.com/opensearch-project/OpenSearch/pull/11775)), [#12034](https://github.com/opensearch-project/OpenSearch/pull/12034))
- Bump `com.avast.gradle:gradle-docker-compose-plugin` from 0.16.12 to 0.17.6 ([#10163](https://github.com/opensearch-project/OpenSearch/pull/10163), [#11692](https://github.com/opensearch-project/OpenSearch/pull/11692))
- Bump `com.squareup.okhttp3:okhttp` from 4.11.0 to 4.12.0 ([#10861](https://github.com/opensearch-project/OpenSearch/pull/10861))
- Bump `org.apache.commons:commons-text` from 1.10.0 to 1.11.0 ([#11344](https://github.com/opensearch-project/OpenSearch/pull/11344))
- Bump `reactor-netty-core` from 1.1.12 to 1.1.13 ([#11350](https://github.com/opensearch-project/OpenSearch/pull/11350))
- Bump `com.gradle.enterprise` from 3.14.1 to 3.16.1 ([#11339](https://github.com/opensearch-project/OpenSearch/pull/11339), [#11629](https://github.com/opensearch-project/OpenSearch/pull/11629))
- Bump `reactor-netty-core` from 1.1.12 to 1.1.15 ([#11350](https://github.com/opensearch-project/OpenSearch/pull/11350)), ([#12042](https://github.com/opensearch-project/OpenSearch/pull/12042))
- Bump `com.gradle.enterprise` from 3.14.1 to 3.16.2 ([#11339](https://github.com/opensearch-project/OpenSearch/pull/11339), [#11629](https://github.com/opensearch-project/OpenSearch/pull/11629), [#12056](https://github.com/opensearch-project/OpenSearch/pull/12056))
- Bump `actions/setup-java` from 3 to 4 ([#11447](https://github.com/opensearch-project/OpenSearch/pull/11447))
- Bump `commons-net:commons-net` from 3.9.0 to 3.10.0 ([#11450](https://github.com/opensearch-project/OpenSearch/pull/11450))
- Bump `org.apache.maven:maven-model` from 3.9.4 to 3.9.6 ([#11445](https://github.com/opensearch-project/OpenSearch/pull/11445))
Expand All @@ -174,12 +176,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Bump `com.maxmind.db:maxmind-db` from 3.0.0 to 3.1.0 ([#11693](https://github.com/opensearch-project/OpenSearch/pull/11693))
- Bump `net.java.dev.jna:jna` from 5.13.0 to 5.14.0 ([#11798](https://github.com/opensearch-project/OpenSearch/pull/11798))
- Bump `lycheeverse/lychee-action` from 1.8.0 to 1.9.1 ([#11795](https://github.com/opensearch-project/OpenSearch/pull/11795), [#11887](https://github.com/opensearch-project/OpenSearch/pull/11887))
- Bump `Lucene` from 9.8.0 to 9.9.1 ([#11421](https://github.com/opensearch-project/OpenSearch/pull/11421))
- Bump `Lucene` from 9.8.0 to 9.9.2 ([#11421](https://github.com/opensearch-project/OpenSearch/pull/11421)), ([#12063](https://github.com/opensearch-project/OpenSearch/pull/12063))
- Bump `com.networknt:json-schema-validator` from 1.0.86 to 1.2.0 ([#11886](https://github.com/opensearch-project/OpenSearch/pull/11886), [#11963](https://github.com/opensearch-project/OpenSearch/pull/11963))
- Bump `com.google.api:gax-httpjson` from 0.103.1 to 2.39.0 ([#11794](https://github.com/opensearch-project/OpenSearch/pull/11794))
- Bump `com.google.oauth-client:google-oauth-client` from 1.34.1 to 1.35.0 ([#11960](https://github.com/opensearch-project/OpenSearch/pull/11960))
- Bump `com.diffplug.spotless` from 6.23.2 to 6.24.0 ([#11962](https://github.com/opensearch-project/OpenSearch/pull/11962))
- Bump `com.diffplug.spotless` from 6.23.2 to 6.25.0 ([#11962](https://github.com/opensearch-project/OpenSearch/pull/11962), [#12055](https://github.com/opensearch-project/OpenSearch/pull/12055))
- Bump `com.google.cloud:google-cloud-core` from 2.5.10 to 2.30.0 ([#11961](https://github.com/opensearch-project/OpenSearch/pull/11961))
- Bump `reactor-core` from 3.5.11 to 3.5.14 ([#12042](https://github.com/opensearch-project/OpenSearch/pull/12042))
- Bump `com.google.http-client:google-http-client-jackson2` from 1.43.3 to 1.44.1 ([#12059](https://github.com/opensearch-project/OpenSearch/pull/12059))
- Bump `peter-evans/create-issue-from-file` from 4 to 5 ([#12057](https://github.com/opensearch-project/OpenSearch/pull/12057))

### Changed
- Mute the query profile IT with concurrent execution ([#9840](https://github.com/opensearch-project/OpenSearch/pull/9840))
Expand Down Expand Up @@ -212,6 +217,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Ensure Jackson default maximums introduced in 2.16.0 do not conflict with OpenSearch settings ([#11890](https://github.com/opensearch-project/OpenSearch/pull/11890))
- Extract cluster management for integration tests into JUnit test rule out of OpenSearchIntegTestCase ([#11877](https://github.com/opensearch-project/OpenSearch/pull/11877)), ([#12000](https://github.com/opensearch-project/OpenSearch/pull/12000))
- Workaround for https://bugs.openjdk.org/browse/JDK-8323659 regression, introduced in JDK-21.0.2 ([#11968](https://github.com/opensearch-project/OpenSearch/pull/11968))
- Updates IpField to be searchable when only `doc_values` are enabled ([#11508](https://github.com/opensearch-project/OpenSearch/pull/11508))

### Deprecated

Expand Down Expand Up @@ -242,6 +248,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Fix tracing context propagation for local transport instrumentation ([#11490](https://github.com/opensearch-project/OpenSearch/pull/11490))
- Fix parsing of single line comments in `lang-painless` ([#11815](https://github.com/opensearch-project/OpenSearch/issues/11815))
- Fix memory leak issue in ReorganizingLongHash ([#11953](https://github.com/opensearch-project/OpenSearch/issues/11953))
- Prevent setting remote_snapshot store type on index creation ([#11867](https://github.com/opensearch-project/OpenSearch/pull/11867))
- [BUG] Fix remote shards balancer when filtering throttled nodes ([#11724](https://github.com/opensearch-project/OpenSearch/pull/11724))

### Security

Expand Down
2 changes: 1 addition & 1 deletion DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ Please follow these formatting guidelines:
* Wildcard imports (`import foo.bar.baz.*`) are forbidden and will cause the build to fail.
* If *absolutely* necessary, you can disable formatting for regions of code with the `// tag::NAME` and `// end::NAME` directives, but note that these are intended for use in documentation, so please make it clear what you have done, and only do this where the benefit clearly outweighs the decrease in consistency.
* Note that JavaDoc and block comments i.e. `/* ... */` are not formatted, but line comments i.e `// ...` are.
* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, if might get called out in PR reviews as something to change.
* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, it might get called out in PR reviews as something to change.

## Adding Dependencies

Expand Down
2 changes: 1 addition & 1 deletion build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ plugins {
id 'lifecycle-base'
id 'opensearch.docker-support'
id 'opensearch.global-build-info'
id "com.diffplug.spotless" version "6.24.0" apply false
id "com.diffplug.spotless" version "6.25.0" apply false
id "org.gradle.test-retry" version "1.5.4" apply false
id "test-report-aggregation"
id 'jacoco-report-aggregation'
Expand Down
10 changes: 5 additions & 5 deletions buildSrc/version.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
opensearch = 3.0.0
lucene = 9.9.1
lucene = 9.9.2

bundled_jdk_vendor = adoptium
bundled_jdk = 21.0.2+13
Expand All @@ -26,12 +26,12 @@ jakarta_annotation = 1.3.5
# when updating the JNA version, also update the version in buildSrc/build.gradle
jna = 5.13.0

netty = 4.1.104.Final
netty = 4.1.106.Final
joda = 2.12.2

# project reactor
reactor_netty = 1.1.13
reactor = 3.5.11
reactor_netty = 1.1.15
reactor = 3.5.14

# client dependencies
httpclient5 = 5.2.1
Expand All @@ -55,7 +55,7 @@ bouncycastle=1.76
randomizedrunner = 2.7.1
junit = 4.13.2
hamcrest = 2.1
mockito = 5.5.0
mockito = 5.10.0
objenesis = 3.2
bytebuddy = 1.14.7

Expand Down
1 change: 0 additions & 1 deletion libs/core/licenses/lucene-core-9.9.1.jar.sha1

This file was deleted.

1 change: 1 addition & 0 deletions libs/core/licenses/lucene-core-9.9.2.jar.sha1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
7699f80220fc80b08413902560904623b88beb9f
4 changes: 2 additions & 2 deletions libs/core/src/main/java/org/opensearch/Version.java
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ public class Version implements Comparable<Version>, ToXContentFragment {
public static final Version V_2_11_0 = new Version(2110099, org.apache.lucene.util.Version.LUCENE_9_7_0);
public static final Version V_2_11_1 = new Version(2110199, org.apache.lucene.util.Version.LUCENE_9_7_0);
public static final Version V_2_11_2 = new Version(2110299, org.apache.lucene.util.Version.LUCENE_9_7_0);
public static final Version V_2_12_0 = new Version(2120099, org.apache.lucene.util.Version.LUCENE_9_9_1);
public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_9_9_1);
public static final Version V_2_12_0 = new Version(2120099, org.apache.lucene.util.Version.LUCENE_9_9_2);
public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_9_9_2);
public static final Version CURRENT = V_3_0_0;

public static Version fromId(int id) {
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
045df3828b6460c032a6551040e31ea432b0aad6

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c5495ba59a627641b3a7c23f6bcb801874c7f7b0

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
025171b63aa1e7a5fd8a7e4e660d6d3110241ea7

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
21a07cdf0fc46b313fe2248f1275cdbdac0ba87b

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
41515e8c51eeaaddceabdb4f86fbc5dbfc25b70e

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
132defb4689f41b51b483b7202b22b6e89fe35fd

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
874c970c4ff958b1140dde52bc17e6a9e7cde662

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e185ae573db04939215f94d6ba869758dcecbde9

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
8fcca44ae16b98e15965093e7696832019fd6f27

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2da179bd95903f0fa73218b8f0d02690c0cfbc94

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
f7e549fdac07140f4cd379a0f517c38434165e95

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
eb19738fd3ca85071fef96978a056a7c94d8e793

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
7e83a83741155ececf9193a4f967e570e170236d

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
dabdea96c7a6d00363b0093a580e7d783efa69a4

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
fb551d9ae6a1bf12bc90a4d26dd8fa3eefac8cb9

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0d495b478861b2d29e0a58d273ca0e6e755292e9

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
9958d813d4cfdd890b4611c679ed36775480fa0d

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
9d4b3315bb625ec2d73fa569fb6bce4589243d5e

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
41515e8c51eeaaddceabdb4f86fbc5dbfc25b70e

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
129f9bd6aa0cf28bfd7d45a8b6a598eed9c67702

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bffc88b7c56d7b553453e4244247a1b0ba1fdc8a

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3970474ed55aa1e3e8de5a5602c342c6b8155371

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2da179bd95903f0fa73218b8f0d02690c0cfbc94

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3221d405ad55a573cf29875a8244a4217cf07185

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c79756fa2dfc28ac81fc9d23a14b17c656c3e560
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@
import com.azure.storage.common.implementation.Constants;
import com.azure.storage.common.policy.RequestRetryOptions;
import com.azure.storage.common.policy.RetryPolicyType;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.opensearch.common.SuppressForbidden;
import org.opensearch.common.regex.Regex;
import org.opensearch.common.settings.MockSecureSettings;
Expand Down Expand Up @@ -188,6 +190,7 @@ protected String requestUniqueId(final HttpExchange exchange) {
@SuppressForbidden(reason = "this test uses a HttpServer to emulate an Azure endpoint")
private static class AzureHTTPStatsCollectorHandler extends HttpStatsCollectorHandler {

private static final Logger testLogger = LogManager.getLogger(AzureHTTPStatsCollectorHandler.class);
private static final Pattern listPattern = Pattern.compile("GET /[a-zA-Z0-9]+\\??.+");
private static final Pattern getPattern = Pattern.compile("GET /[^?/]+/[^?/]+\\??.*");

Expand All @@ -197,6 +200,7 @@ private AzureHTTPStatsCollectorHandler(HttpHandler delegate) {

@Override
protected void maybeTrack(String request, Headers headers) {
testLogger.info(request, headers);
if (getPattern.matcher(request).matches()) {
trackRequest("GetBlob");
} else if (Regex.simpleMatch("HEAD /*/*", request)) {
Expand Down
2 changes: 1 addition & 1 deletion plugins/repository-gcs/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ dependencies {
api 'com.google.http-client:google-http-client:1.43.3'
api 'com.google.http-client:google-http-client-appengine:1.43.3'
api 'com.google.http-client:google-http-client-gson:1.43.3'
api 'com.google.http-client:google-http-client-jackson2:1.43.3'
api 'com.google.http-client:google-http-client-jackson2:1.44.1'

api 'com.google.oauth-client:google-oauth-client:1.34.1'

Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3f1947de0fd9eb250af16abe6103c11e68d11635

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6ec5bd2be65d5529e58e9e482b747c1135b3736b

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c5495ba59a627641b3a7c23f6bcb801874c7f7b0

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
025171b63aa1e7a5fd8a7e4e660d6d3110241ea7

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
21a07cdf0fc46b313fe2248f1275cdbdac0ba87b

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
41515e8c51eeaaddceabdb4f86fbc5dbfc25b70e

This file was deleted.

Loading

0 comments on commit d1e90a1

Please sign in to comment.