Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SDK] Add support for createComponent extension points #2981

Closed
1 task done
owaiskazi19 opened this issue Apr 19, 2022 · 6 comments
Closed
1 task done

[SDK] Add support for createComponent extension points #2981

owaiskazi19 opened this issue Apr 19, 2022 · 6 comments
Assignees
Labels
Build Libraries & Interfaces enhancement Enhancement or improvement to existing feature or request

Comments

@owaiskazi19
Copy link
Member

owaiskazi19 commented Apr 19, 2022

Is your feature request related to a problem? Please describe.

For every extension points, SDK should have a TransportRequest to communicate with OpenSearch. This issue focuses on the below extension point:

This covers

  1. clusterState
  2. clusterSettings
  3. LocalNode

This extensions will also cover a part of communication of sql plugin with OpenSearch and make the plugin extensible.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@owaiskazi19
Copy link
Member Author

The workflow to create interface for any extension point is:

  1. OpenSearch will send a request to initialize all the plugins listening on a pre-defined port.
  2. Once the initialization is successful then OpenSearch will send a request to see if any of the the plugin requires the extension points present in the SDK.
  3. If it does, then a TransportRequest will be send from OpenSearch to the plugin which is registered by the SDK.
  4. After the registration, the following extension point will be invoke on the OpenSearch end and an acknowledgment will be send to the plugin to mention that the work has been done.

Steps to achieve the above workflow:

  1. Create a TransportRequest for the extension point which initializes the required parameters in OpenSearch.
  2. Implement the extension point in PluginOrchestrator
  3. Register the TransportRequest in plugin.
  4. Create a TransportResponse for the extension point to send the acknowledgment to the extension.

@saratvemulapalli saratvemulapalli changed the title [POC] Implement TransportRequest for the extension points on SDK [POC] Add support for additional extension points on OpenSearch SDK Apr 27, 2022
@saratvemulapalli saratvemulapalli changed the title [POC] Add support for additional extension points on OpenSearch SDK [SDK] Add support for additional extension points Apr 27, 2022
@owaiskazi19
Copy link
Member Author

Design for createComponent with SQL plugin.

SQL plugin requires ClusterState and ClusterSettings for createComponent extension point.
The design needs to be change for this extension point. SQL plugin can send a TransportRequest to OpenSearch to get the ClusterState and ClusterSettings as ClusterState keeps on changing in OpenSearch. OpenSearch will register the request and provide the ClusterState and ClusterSettings as response to SQL plugin.

@owaiskazi19
Copy link
Member Author

owaiskazi19 commented May 10, 2022

As mentioned in the above comment, because of the new design we can try using sendRequest from RemoteConnectionManager to send a TransportRequest from SDK to OpenSearch to add the node in the RemoteConnectionManager map and check if the error mentioned here is resolved.

@owaiskazi19 owaiskazi19 changed the title [SDK] Add support for additional extension points [SDK] Add support for createComponent extension points May 10, 2022
@owaiskazi19 owaiskazi19 self-assigned this May 18, 2022
@owaiskazi19
Copy link
Member Author

owaiskazi19 commented May 20, 2022

Changing the node here to localNode of OpenSearch resolved the error. Moved the sendRequest in handlePluginRequest for a better approach.
Currently looking into

Caused by: java.lang.IllegalStateException: Message not fully read (request) for requestId [3], action [internal:discovery/createcomponent], available [0]; resetting
        at org.opensearch.transport.InboundHandler.checkStreamIsFullyConsumed(InboundHandler.java:311) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.InboundHandler.handleRequest(InboundHandler.java:239) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.InboundHandler.messageReceived(InboundHandler.java:127) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:109) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:761) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:170) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:145) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:110) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]

@owaiskazi19
Copy link
Member Author

Resolved the above issue. It was mainly because of using Enum in the CreateComponentRequest. Currently looking into opensearch-project/opensearch-sdk-java#6

@owaiskazi19
Copy link
Member Author

Resolved opensearch-project/opensearch-sdk-java#6. The issue was because of creating a new transport object for the new CreateComponentRequest which resulted in a new ConnectionManagerMap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Libraries & Interfaces enhancement Enhancement or improvement to existing feature or request
Projects
None yet
Development

No branches or pull requests

1 participant