Deprecate types in index API #36575

mayya-sharipova · 2018-12-12T23:10:24Z

deprecate type-based constructors of IndexRequest
update tests to use typeless IndexRequest constructors
no yaml tests as they have been already added in Make typeless APIs usable with indices whose type name is different from _doc #35790

Relates to #35190

elasticmachine · 2018-12-12T23:10:46Z

Pinging @elastic/es-search

- deprecate type-based constructors of IndexRequest - update tests to use typeless IndexRequest constructors - no yaml tests as they have been already added in elastic#35790 Relates to elastic#35190

markharwood

Added a few minor comments but otherwise LGTM

markharwood · 2018-12-14T10:03:24Z

server/src/main/java/org/elasticsearch/rest/action/document/RestIndexAction.java

+        LogManager.getLogger(RestDeleteAction.class));
+    public static final String TYPES_DEPRECATION_MESSAGE = "[types removal] Specifying types in " +
+        "document index requests is deprecated, use the /{index}/_doc/{id} endpoint instead.";
+


This message may be misleading if the user didn't supply an ID.

markharwood · 2018-12-14T10:04:20Z

server/src/main/java/org/elasticsearch/rest/action/document/RestIndexAction.java

-            throw new IllegalArgumentException("You may only use the [include_type_name=false] option with the index APIs with the " +
-                    "[{index}/_doc/{id}] and [{index}/_doc] endpoints.");
+        if (!type.equals(MapperService.SINGLE_MAPPING_NAME)) {
+            deprecationLogger.deprecatedAndMaybeLog("index_with_types", TYPES_DEPRECATION_MESSAGE);


For readability we prefer X==false rather than !X

markharwood · 2018-12-17T09:33:41Z

...igh-level/src/test/java/org/elasticsearch/client/documentation/MigrationDocumentationIT.java

@@ -82,7 +82,7 @@ public void testRequests() throws Exception {
        RestHighLevelClient client = highLevelClient();
        {
            //tag::migration-request-ctor
-            IndexRequest request = new IndexRequest("index", "_doc", "id"); // <1>
+            IndexRequest request = new IndexRequest("index").id("id"); // <1>
            request.source("{\"field\":\"value\"}", XContentType.JSON);


Related doc says Create an IndexRequest using its constructor -perhaps add comment on the use of id() setter too.

jtibshirani

Looks good to me too, just a few small questions!

jtibshirani · 2018-12-17T18:10:58Z

...-clusters/src/test/java/org/elasticsearch/search/CrossClusterSearchUnavailableClusterIT.java

@@ -135,7 +135,7 @@ public void testSearchSkipUnavailable() throws IOException {

            for (int i = 0; i < 10; i++) {
                restHighLevelClient.index(
-                        new IndexRequest("index", "doc", String.valueOf(i)).source("field", "value"), RequestOptions.DEFAULT);
+                        new IndexRequest("index", "_doc", String.valueOf(i)).source("field", "value"), RequestOptions.DEFAULT);


For my knowledge, why didn't we remove the type altogether, as in new IndexRequest("index").id(...))?

jtibshirani · 2018-12-17T18:15:15Z

...k/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/cli/CliIntegrationTestCase.java

@@ -59,6 +60,7 @@ protected SecurityConfig securityConfig() {
    protected void index(String index, CheckedConsumer<XContentBuilder, IOException> body) throws IOException {
        Request request = new Request("PUT", "/" + index + "/doc/1");


I'm wondering if we could we switch this to _doc instead of expecting a warning?

@jtibshirani Thanks Julie. Changing this into a _doc in this file or in JdbcIntegrationTestCase.java now will lead to failures of many sql tests. I am planning to have a subsequent PR that can go through our xpack products and change them to use non-deprecated versions.

Making these changes in a follow-up PR works for me. I'll add a note to the meta issue to track this. It could be that the individual teams should make these changes, as I think it may require more than just updating integration tests. For example, the security indices are still created using the old type name doc.

jtibshirani · 2018-12-17T18:15:26Z

...plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcIntegrationTestCase.java

@@ -84,6 +85,7 @@ public static void index(String index, CheckedConsumer<XContentBuilder, IOExcept

    public static void index(String index, String documentId, CheckedConsumer<XContentBuilder, IOException> body) throws IOException {
        Request request = new Request("PUT", "/" + index + "/doc/" + documentId);
+        request.setOptions(expectWarnings(RestIndexAction.TYPES_DEPRECATION_MESSAGE));


Same question here around switching to _doc.

jtibshirani · 2018-12-17T18:19:18Z

...est-high-level/src/test/java/org/elasticsearch/client/BulkRequestWithGlobalParametersIT.java

-    }
-
-    @SuppressWarnings("unchecked")
-    public void testTypeGlobalAndPerRequest() throws IOException {


It might be nice to leave this in (with expectWarnings calls), as I don't see this functionality being tested elsewhere.

@jtibshirani Thanks for the suggestion, Julie. But this test will not work anymore, as creating an IndexRequest without a type like new IndexRequest("index2").id("2") will assign a type of _doc to it. While the test here checks that that that the result type should be "global_type".

I am going to temporary disable this test with AwaitsFix, and let @markharwood address this test through his bulk API change.

Oh, I didn't realize this was actually broken. I now see it is also broken for update and delete. I would also be okay adding AwaitsFix, and we can work to fix the issue as soon as possible in another PR.

…ndex

mayya-sharipova · 2018-12-18T11:56:00Z

@elasticmachine run the gradle build tests 1

mayya-sharipova · 2018-12-18T11:58:54Z

@elasticmachine run the default distro tests

* elastic/master: (31 commits) enable bwc tests and switch transport serialization version to 6.6.0 for CAS features [DOCs] Adds ml-cpp PRs to alpha release notes (elastic#36790) Synchronize WriteReplicaResult callbacks (elastic#36770) Add CcrRestoreSourceService to track sessions (elastic#36578) [Painless] Add tests for boxed return types (elastic#36747) Internal: Remove originalSettings from Node (elastic#36569) [ILM][DOCS] Update ILM API authorization docs (elastic#36749) Core: Deprecate use of scientific notation in epoch time parsing (elastic#36691) [ML] Merge the Jindex master feature branch (elastic#36702) Tests: Mute SnapshotDisruptionIT.testDisruptionOnSnapshotInitialization Update versions in SearchSortValues transport serialization Update version in SearchHits transport serialization [Geo] Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach (elastic#36751) [Docs] Fix error in Common Grams Token Filter (elastic#36774) Fix rollup search statistics (elastic#36674) SQL: Fix wrong appliance of StackOverflow limit for IN (elastic#36724) [TEST] Added more logging Invalidate Token API enhancements - HLRC (elastic#36362) Deprecate types in index API (elastic#36575) Disable bwc tests until elastic#36555 backport is complete (elastic#36737) ...

mayya-sharipova added >deprecation v7.0.0 :Search Foundations/Mapping Index mappings, including merging and defining field types labels Dec 12, 2018

mayya-sharipova requested review from jtibshirani and markharwood December 12, 2018 23:10

mayya-sharipova added the WIP label Dec 13, 2018

mayya-sharipova force-pushed the deprecate_types_index branch 3 times, most recently from 06a9045 to 5de4d1b Compare December 14, 2018 23:18

mayya-sharipova removed the WIP label Dec 14, 2018

Deprecate types in index API

560315b

- deprecate type-based constructors of IndexRequest - update tests to use typeless IndexRequest constructors - no yaml tests as they have been already added in elastic#35790 Relates to elastic#35190

mayya-sharipova force-pushed the deprecate_types_index branch from 5de4d1b to 560315b Compare December 16, 2018 17:49

markharwood approved these changes Dec 17, 2018

View reviewed changes

Address Mark's comments

60f0410

jtibshirani approved these changes Dec 17, 2018

View reviewed changes

Change deprecated api in IndexRequestTests

1f82877

jtibshirani mentioned this pull request Dec 17, 2018

Implementation tracking for 7.0 types deprecation. #35190

Closed

48 tasks

mayya-sharipova added 4 commits December 17, 2018 15:09

Address Julie's comments

5f161bd

Merge remote-tracking branch 'upstream/master' into deprecate_types_i…

a35d4b5

…ndex

Address test failures

0a9e6bd

Merge remote-tracking branch 'upstream/master' into deprecate_types_i…

639729b

…ndex

mayya-sharipova merged commit f884b2b into elastic:master Dec 18, 2018

colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate types in index API #36575

Deprecate types in index API #36575

mayya-sharipova commented Dec 12, 2018

elasticmachine commented Dec 12, 2018

markharwood left a comment

markharwood Dec 14, 2018

markharwood Dec 14, 2018

markharwood Dec 17, 2018

jtibshirani left a comment

jtibshirani Dec 17, 2018

jtibshirani Dec 17, 2018

mayya-sharipova Dec 17, 2018

jtibshirani Dec 17, 2018

jtibshirani Dec 17, 2018

jtibshirani Dec 17, 2018

mayya-sharipova Dec 17, 2018

mayya-sharipova Dec 17, 2018

jtibshirani Dec 17, 2018

mayya-sharipova commented Dec 18, 2018

mayya-sharipova commented Dec 18, 2018

		@@ -59,6 +60,7 @@ protected SecurityConfig securityConfig() {
		protected void index(String index, CheckedConsumer<XContentBuilder, IOException> body) throws IOException {
		Request request = new Request("PUT", "/" + index + "/doc/1");

Deprecate types in index API #36575

Deprecate types in index API #36575

Conversation

mayya-sharipova commented Dec 12, 2018

elasticmachine commented Dec 12, 2018

markharwood left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtibshirani left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mayya-sharipova commented Dec 18, 2018

mayya-sharipova commented Dec 18, 2018