Skip to content

Commit

Permalink
Tweak MD design doc
Browse files Browse the repository at this point in the history
Signed-off-by: Kristen Tian <tyarong@amazon.com>
  • Loading branch information
kristenTian committed Nov 2, 2022
1 parent 1ecfee7 commit f77236a
Showing 1 changed file with 10 additions and 10 deletions.
20 changes: 10 additions & 10 deletions docs/multi-datasource/client_management_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ This design is part of the OpenSearch Dashboards multi data source project [[RFC

### 4.0 Answer some critical design questions

**1.** **How to set up connection(clients) for different datasources?**
Similar to how current OpenSearch Dashboards talks to default OS by creating opensearch node.js client using [opensearch-js](https://github.com/opensearch-project/opensearch-js) library, for datasources we also create clients for each. Critical params that differentiate data sources are `url` and `auth`
**1.** **How to set up connection(clients) for different data sources?**
Similar to how current OpenSearch Dashboards talks to default OS by creating opensearch node.js client using [opensearch-js](https://github.com/opensearch-project/opensearch-js) library, for data sources we also create clients for each. Critical params that differentiate data sources are `url` and `auth`

```ts
const { Client } = require('@opensearch-project/opensearch');
Expand Down Expand Up @@ -77,14 +77,14 @@ The context is that user can only turn on/off multiple datasource feature by upd

**4.How to manage multiple clients/connection efficiently, and not consume all the memory?**

- For datasources with different endpoint, user client Pooling (E.g. LRU cache)
- For data sources with different endpoint, user client Pooling (E.g. LRU cache)
- For data sources with same endpoint, but different user, use connection pooling strategy (child client) provided by opensearch-js.

**5.Where should we implement the core logic?**
Current `opensearch service` exists in core. The module we'll implement has similarity function wise, but we choose to implement `data source service` in plugin along with `crypto` service for the following reasons.

1. Data source is a feature that can be turned on or off. Plugin is born for such plugable use case.
2. We don't mess up with OpenSearch Dashboards core, since this is an experimental feature, the potential risk of breaking existing behavior will be lower if we use plugin. Worst case, user could just uninstall the plugin.
1. Data source is a feature that can be turned on or off. Plugin is born for such pluggable use case.
2. We don't mess up with OpenSearch Dashboards core, since this is an experimental feature, the potential risk of breaking existing behavior will be lowered if we use plugin. Worst case, user could just uninstall the plugin.
3. Complexity wise, it's about the same amount of work.

### 4.1 Data Source Plugin
Expand Down Expand Up @@ -130,14 +130,14 @@ We need to configure the data source client by either creating a new one, or loo
}
```

- Get root client: Look up client Pool by **endpoint**, return client if existed. If misses, we create new client instance and load into pool. At this step, the client won't have any auth info.
- Get root client: Look up client Pool by **endpoint**, return client if existed. If it misses, we create new client instance and load into pool. At this step, the client won't have any auth info.

- Get credentials: Call crypto service utilities to **decrypt** user credentials from `DataSource` Object.
- Assemble the actual query client: With auth info and root client, well leverage the openearch-js connection pooling strategy to create the actual query client from root client by `client.child()`.
- Assemble the actual query client: With auth info and root client, well leverage the `opensearch-js` connection pooling strategy to create the actual query client from root client by `client.child()`.

#### 4.2.1 Legacy Client

OpenSearch Dashboards is forked from Kibana 7.10. At the time of the fork happened, there are 2 types of client used in the codebase. One is the new client, which later was migrated as `opensearhc-js`, the other one is the legacy client which is `elasticsearc-js`. Legacy clients are still used many critical features, such as visualization, index pattern management, along with new client.
OpenSearch Dashboards is forked from Kibana 7.10. At the time of the fork happened, there are 2 types of client used in the codebase. One is the new client, which later was migrated as `opensearch-js`, the other one is the legacy client which is `elasticsearch-js`. Legacy clients are still used many critical features, such as visualization, index pattern management, along with new client.

```ts
// legacy client
Expand Down Expand Up @@ -174,7 +174,7 @@ This is for plugin to access data source client via request handler. For example

### 4.4 Refactor data plugin search module to call core API to get datasource client

`Search strategy` is the low level API of data plugin search module. It retrieve clients and query OpenSearch. It needs to be refactored to switch between default client and datasource client, depending on whether a request is send to datasource or not.
`Search strategy` is the low level API of data plugin search module. It retrieves clients and query OpenSearch. It needs to be refactored to switch between default client and datasource client, depending on whether a request is send to datasource or not.

Currently default client is retrieved by search module of data plugin to interact with OpenSearch by this API call. Ref: [opensearch-search-strategy.ts](https://github.com/opensearch-project/opensearch-dashboards/blob/e3b34df1dea59a253884f6da4e49c3e717d362c9/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.ts#L75)

Expand All @@ -184,7 +184,7 @@ const client: OpenSearchClient = core.opensearch.client.asCurrentUser;
client.search(params);
```

Similarly well have the following for datasource use case. `AsCurrentUser` is something doesnt make sense for datasource, because its always thecurrentuser credential defined in thedatasource”, that we are using to create the client, or look up the client pool.
Similarly well have the following for datasource use case. `AsCurrentUser` is something does not make sense for datasource, because its always thecurrentuser credential defined in thedatasource”, that we are using to create the client, or look up the client pool.

```ts
if (request.dataSource) {
Expand Down

0 comments on commit f77236a

Please sign in to comment.