Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for ML neural search. #504

Merged
merged 1 commit into from
Aug 13, 2024

Conversation

dblock
Copy link
Member

@dblock dblock commented Aug 13, 2024

Description

This completes end-to-end flow for neural search.

Added /_plugins/_ml/models/{model_id}/_deploy, _undeploy and knn_vector type to passage_embedding.

Issues Resolved

Closes #315.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

github-actions bot commented Aug 13, 2024

Changes Analysis

Commit SHA: 392a0b6
Comparing To SHA: 42fd4d7

API Changes

Summary

├─┬Paths
│ ├──[➕] path (3857:3)
│ └──[➕] path (3842:3)
└─┬Components
  ├──[➕] responses (25705:7)
  ├──[➕] responses (25657:7)
  ├──[➕] parameters (19578:7)
  ├──[➕] parameters (19560:7)
  ├──[➕] schemas (45857:7)
  ├──[➕] schemas (36832:7)
  ├──[➕] schemas (35230:7)
  ├──[➕] schemas (35204:7)
  ├──[➕] schemas (36826:7)
  ├──[➕] schemas (45848:7)
  ├──[➕] schemas (35219:7)
  ├──[➕] schemas (45852:7)
  ├─┬_common.query_dsl:QueryContainer
  │ └──[➕] properties (37098:9)
  ├─┬ml._common:Task
  │ └──[➕] properties (45843:9)
  └─┬_common.mapping:Property
    ├──[➕] oneOf (35065:7)
    ├─┬ONEOF
    │ └──[🔀] $ref (35220:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35470:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35154:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34762:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35253:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35263:13)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35838:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35292:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34613:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34643:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35748:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35609:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34815:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35093:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35117:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34681:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35240:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35827:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35654:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34846:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35027:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34994:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35814:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35303:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35590:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34712:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34656:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34981:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34735:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34952:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34788:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35175:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35143:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34918:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35106:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35548:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34624:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35010:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35483:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35358:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35342:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (34828:9)❌ 
    ├─┬ONEOF
    │ └──[🔀] $ref (35701:9)❌ 
    └─┬ONEOF
      └──[🔀] $ref (35053:9)❌ 

Document Element Total Changes Breaking Changes
paths 2 0
components 59 44
  • BREAKING Changes: 44 out of 61
  • Modifications: 44
  • Additions: 17
  • Breaking Modifications: 44

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/10371987146/artifacts/1807014485

API Coverage

Before After Δ
Covered (%) 522 (51.13 %) 524 (51.32 %) 2 (0.19 %)
Uncovered (%) 499 (48.87 %) 497 (48.68 %) -2 (-0.19 %)
Unknown 26 26 0

Copy link

Spec Test Coverage Analysis

Total Tested
550 212 (38.55 %)

@dblock dblock force-pushed the neural-search branch 2 times, most recently from 783a5ae to 8909bfe Compare August 13, 2024 14:43
Signed-off-by: dblock <dblock@amazon.com>
@@ -59,6 +59,7 @@ gsub
Gsub
haasephonetik
heteroscedastic
hnsw
homoscedastic
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've learned a big word today 😄

@nhtruong nhtruong merged commit 09824d0 into opensearch-project:main Aug 13, 2024
15 checks passed
@dblock dblock deleted the neural-search branch August 13, 2024 15:15
@dblock dblock mentioned this pull request Aug 13, 2024
40 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants