Add data source signing support

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>

CHANGELOG.md

@@ -1,7 +1,9 @@
 # CHANGELOG
+
 Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ## [Unreleased]
+
 ### 💥 Breaking Changes
 
 ### Deprecations
@@ -10,18 +12,20 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ### 📈 Features/Enhancements
 
-* [MD] Support legacy client for data source ([#2204](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2204))
-* [Plugin Helpers] Facilitate version changes ([#2398](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2398))
+- [MD] Support legacy client for data source ([#2204](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2204))
+- [MD] Add data source signing support ([#2510](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2510))
+- [Plugin Helpers] Facilitate version changes ([#2398](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2398))
 
 ### 🐛 Bug Fixes
-* [Vis Builder] Fixes auto bounds for timeseries bar chart visualization ([2401](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2401))
-* [Vis Builder] Fixes visualization shift when editing agg ([2401](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2401))
-* [Vis Builder] Renames "Histogram" to "Bar" in vis type picker ([2401](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2401))
-* [MD] Add data source param to low-level search call in Discover ([#2431](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2431))
+
+- [Vis Builder] Fixes auto bounds for timeseries bar chart visualization ([2401](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2401))
+- [Vis Builder] Fixes visualization shift when editing agg ([2401](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2401))
+- [Vis Builder] Renames "Histogram" to "Bar" in vis type picker ([2401](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2401))
+- [MD] Add data source param to low-level search call in Discover ([#2431](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2431))
 
 ### 🚞 Infrastructure
 
-* Add CHANGELOG.md and related workflows ([#2414](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2414))
+- Add CHANGELOG.md and related workflows ([#2414](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2414))
 
 ### 📝 Documentation
 
@@ -32,56 +36,56 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 ### 🔩 Tests
 
 ## [2.x]
+
 ### 💥 Breaking Changes
 
 ### Deprecations
 
 ### 🛡 Security
 
-* Use a forced CSP-compliant interpreter with Vega visualizations ([#2352](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2352))
-* Bump moment-timezone from 0.5.34 to 0.5.37 ([#2361](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2361))
-* [CVE-2022-33987] Upgrade geckodriver to 3.0.2 ([#2166](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2166))
-* Bumps percy-agent to use non-beta version ([#2415](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2415))
-* Resolve sub-dependent d3-color version and potential security issue ([#2454](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2454))
+- Use a forced CSP-compliant interpreter with Vega visualizations ([#2352](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2352))
+- Bump moment-timezone from 0.5.34 to 0.5.37 ([#2361](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2361))
+- [CVE-2022-33987] Upgrade geckodriver to 3.0.2 ([#2166](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2166))
+- Bumps percy-agent to use non-beta version ([#2415](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2415))
+- Resolve sub-dependent d3-color version and potential security issue ([#2454](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2454))
 
 ### 📈 Features/Enhancements
 
-* Add updated_at column to objects' tables ([#1218](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/1218)) 
-* [Viz Builder] State validation before dispatching and loading ([#2351](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2351))
-* [Viz Builder] Create a new wizard directly on a dashboard ([#2384](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2384))
+- Add updated_at column to objects' tables ([#1218](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/1218))
+- [Viz Builder] State validation before dispatching and loading ([#2351](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2351))
+- [Viz Builder] Create a new wizard directly on a dashboard ([#2384](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2384))
 
 ### 🐛 Bug Fixes
 
-* [Viz Builder] Fixes time series for new chart types ([#2309](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2309))
-* [Viz Builder] Add index pattern info when loading embeddable ([#2363](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2363))
-* Fixes management app breadcrumb error ([#2344](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2344))
+- [Viz Builder] Fixes time series for new chart types ([#2309](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2309))
+- [Viz Builder] Add index pattern info when loading embeddable ([#2363](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2363))
+- Fixes management app breadcrumb error ([#2344](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2344))
 
 ### 🚞 Infrastructure
 
-* Add path ignore for markdown files for CI ([#2312](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2312))
-* Updating WS scans to ignore BWC artifacts in `cypress` ([#2408](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2408))
-
+- Add path ignore for markdown files for CI ([#2312](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2312))
+- Updating WS scans to ignore BWC artifacts in `cypress` ([#2408](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2408))
 
 ### 📝 Documentation
 
-* README.md for saving index pattern relationship ([#2276](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2276))
-* Remove extra typo from README. ([#2403](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2403))
-* Add sample config for multi data source feature in yml template. ([#2428](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2428))
-* README.md for dataSource and dataSourceManagement Plugin ([#2448](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2448))
+- README.md for saving index pattern relationship ([#2276](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2276))
+- Remove extra typo from README. ([#2403](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2403))
+- Add sample config for multi data source feature in yml template. ([#2428](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2428))
+- README.md for dataSource and dataSourceManagement Plugin ([#2448](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2448))
 
 ### 🛠 Maintenance
 
-* Increment from 2.3 to 2.4. ([#2295](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2295))
-* Adding @zengyan-amazon as maintainer ([#2419](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2419))
-* Updating @tmarkley to Emeritus status. ([#2423](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2423))
-* Adding sample config for multi data source in yml config template. ([#2428](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2428))
-* Adding @kristenTian as maintainer. ([#2450](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2450))
+- Increment from 2.3 to 2.4. ([#2295](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2295))
+- Adding @zengyan-amazon as maintainer ([#2419](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2419))
+- Updating @tmarkley to Emeritus status. ([#2423](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2423))
+- Adding sample config for multi data source in yml config template. ([#2428](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2428))
+- Adding @kristenTian as maintainer. ([#2450](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2450))
 
 ### 🪛 Refactoring
 
 ### 🔩 Tests
 
-* Update caniuse to fix failed integration tests ([#2322](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2322))
+- Update caniuse to fix failed integration tests ([#2322](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2322))
 
-[Unreleased]: https://github.com/opensearch-project/OpenSearch-Dashboards/compare/2.3.0...HEAD
+[unreleased]: https://github.com/opensearch-project/OpenSearch-Dashboards/compare/2.3.0...HEAD
 [2.x]: https://github.com/opensearch-project/OpenSearch-Dashboards/compare/2.3.0...2.x

src/plugins/data_source/README.md

@@ -5,50 +5,61 @@ An OpenSearch Dashboards plugin
 This plugin introduces support for multiple data sources into OpenSearch Dashboards and provides related functions to connect to OpenSearch data sources.
 
 ## Configuration
+
 Update the following configuration in the `opensearch_dashboards.yml` file to apply changes. Refer to the schema [here](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data_source/config.ts) for supported configurations.
 
 1. The dataSource plugin is disabled by default; to enable it:
-`data_source.enabled: true`
+   `data_source.enabled: true`
 
 2. The audit trail is enabled by default for logging the access to data source; to disable it:
-`data_source.audit.enabled: false`
+   `data_source.audit.enabled: false`
+
+- Current auditor configuration:
 
- - Current auditor configuration:
 ```
 data_source.audit.appender.kind: 'file'
 data_source.audit.appender.layout.kind: 'pattern'
 data_source.audit.appender.path: '/tmp/opensearch-dashboards-data-source-audit.log'
 ```
 
 3. The default encryption-related configuration parameters are:
+
 ```
 data_source.encryption.wrappingKeyName: 'changeme'
 data_source.encryption.wrappingKeyNamespace: 'changeme'
 data_source.encryption.wrappingKey: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
 ```
+
 Note that if any of the encryption keyring configuration values change (wrappingKeyName/wrappingKeyNamespace/wrappingKey), none of the previously-encrypted credentials can be decrypted; therefore, credentials of previously created data sources must be updated to continue use.
 
 **What are the best practices for generating a secure wrapping key?**  
 WrappingKey is an array of 32 random numbers. Read [more](https://en.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator) about best practices for generating a secure wrapping key.
 
 ## Public
+
 The public plugin is used to enable and disable the features related to multi data source available in other plugins. e.g. data_source_management, index_pattern_management
 
 - Add as a required dependency for whole plugin on/off switch
 - Add as opitional dependency for partial flow changes control
 
 ## Server
+
 The provided data source client is integrated with default search strategy in data plugin. When data source id presented in IOpenSearchSearchRequest, data source client will be used.
 
 ### Data Source Service
-The data source service will provide a data source client given a data source id and optional client configurations. 
+
+The data source service will provide a data source client given a data source id and optional client configurations.
 
 Currently supported client config is:
+
 - `data_source.clientPool.size`
 
 Data source service uses LRU cache to cache the root client to improve client pool usage.
+
 #### Example usage:
+
 In the RequestHandler, get an instance of the client using:
+
 ```ts
 client: OpenSearchClient = await context.dataSource.opensearch.getClient(dataSourceId);
 
@@ -57,17 +68,19 @@ apiCaller: LegacyAPICaller = context.dataSource.opensearch.legacy.getClient(data
 ```
 
 ### Data Source Client Wrapper
+
 The data source saved object client wrapper overrides the write related action for data source object in order to perform validation and encryption actions of the authentication information inside data source.
 
-### Cryptography Client
-The research for choosing a suitable stack can be found in: [#1756](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1756)
-#### Example usage:
-```ts
-//Encrypt
-const encryptedPassword = await this.cryptographyClient.encryptAndEncode(password);
-//Decrypt
-const decodedPassword = await this.cryptographyClient.decodeAndDecrypt(password);
-```
+### Cryptography service
+
+The cryptography service performs encryption / decryption on data source credentials (support no_auth and username_password for now). Highlight the following security best practices (see more details on https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1756)
+
+a. Envelope encryption - has multiple benefits including strong protection on data keys, encryption the same data with multiple wrappign keys, etc
+
+b. Key derivation algorithm - HKDF with SHA-384, which “helps you avoid accidental reuse of a data encryption key and reduces the risk of overusing a data key.”
+
+c. Signature algorithm - ECDSA with P-384 and SHA-384. Under multiple data source case, data source indices stored on OpenSearch can be modified / replaced by attacker. With ECDSA signature, ciphertext decryption will fail if it’s getting pullted. No one will be able to create another signature that verifies with the public key because the private key has been dropped.
+
 ---
 
 ## Development

src/plugins/data_source/server/client/configure_client.test.ts

@@ -14,17 +14,18 @@ import { configureClient } from './configure_client';
 import { ClientOptions } from '@opensearch-project/opensearch';
 // eslint-disable-next-line @osd/eslint/no-restricted-paths
 import { opensearchClientMock } from '../../../../core/server/opensearch/client/mocks';
-import { CryptographyClient } from '../cryptography';
+import { cryptographyServiceSetupMock } from '../cryptography_service.mocks';
+import { CryptographyServiceSetup } from '../cryptography_service';
 import { DataSourceClientParams } from '../types';
 
 const DATA_SOURCE_ID = 'a54b76ec86771ee865a0f74a305dfff8';
-const cryptoClient = new CryptographyClient('test', 'test', new Array(32).fill(0));
 
 // TODO: improve UT
 describe('configureClient', () => {
   let logger: ReturnType<typeof loggingSystemMock.createLogger>;
   let config: DataSourcePluginConfigType;
   let savedObjectsMock: jest.Mocked<SavedObjectsClientContract>;
+  let cryptographyMock: jest.Mocked<CryptographyServiceSetup>;
   let clientPoolSetup: OpenSearchClientPoolSetup;
   let clientOptions: ClientOptions;
   let dataSourceAttr: DataSourceAttributes;
@@ -35,6 +36,8 @@ describe('configureClient', () => {
     dsClient = opensearchClientMock.createInternalClient();
     logger = loggingSystemMock.createLogger();
     savedObjectsMock = savedObjectsClientMock.create();
+    cryptographyMock = cryptographyServiceSetupMock.create();
+
     config = {
       enabled: true,
       clientPool: {
@@ -75,7 +78,7 @@ describe('configureClient', () => {
     dataSourceClientParams = {
       dataSourceId: DATA_SOURCE_ID,
       savedObjects: savedObjectsMock,
-      cryptographyClient: cryptoClient,
+      cryptography: cryptographyMock,
     };
 
     ClientMock.mockImplementation(() => dsClient);
@@ -109,14 +112,53 @@ describe('configureClient', () => {
     expect(client).toBe(dsClient.child.mock.results[0].value);
   });
 
-  test('configure client with auth.type == username_password, will first call decrypt()', async () => {
-    const spy = jest.spyOn(cryptoClient, 'decodeAndDecrypt').mockResolvedValue('password');
+  test('configure client with auth.type == username_password, will first call decodeAndDecrypt()', async () => {
+    const decodeAndDecryptSpy = jest.spyOn(cryptographyMock, 'decodeAndDecrypt').mockResolvedValue({
+      decryptedText: 'password',
+      encryptionContext: { endpoint: 'http://localhost' },
+    });
 
     const client = await configureClient(dataSourceClientParams, clientPoolSetup, config, logger);
 
     expect(ClientMock).toHaveBeenCalledTimes(1);
     expect(savedObjectsMock.get).toHaveBeenCalledTimes(1);
-    expect(spy).toHaveBeenCalledTimes(1);
+    expect(decodeAndDecryptSpy).toHaveBeenCalledTimes(1);
     expect(client).toBe(dsClient.child.mock.results[0].value);
   });
+
+  test('configure client with auth.type == username_password and password contaminated', async () => {
+    const expectedErrMsg =
+      'Encrypted "auth.credentials.password" contaminated. Please delete and create another data source.';
+
+    const decodeAndDecryptSpy = jest
+      .spyOn(cryptographyMock, 'decodeAndDecrypt')
+      .mockImplementation(() => {
+        throw new Error(expectedErrMsg);
+      });
+
+    await expect(
+      configureClient(dataSourceClientParams, clientPoolSetup, config, logger)
+    ).rejects.toThrowError(expectedErrMsg);
+
+    expect(ClientMock).toHaveBeenCalledTimes(1);
+    expect(savedObjectsMock.get).toHaveBeenCalledTimes(1);
+    expect(decodeAndDecryptSpy).toHaveBeenCalledTimes(1);
+  });
+
+  test('configure client with auth.type == username_password and endpoint contaminated', async () => {
+    const expectedErrMsg =
+      'Data source "endpoint" contaminated. Please delete and create another data source.';
+    const decodeAndDecryptSpy = jest.spyOn(cryptographyMock, 'decodeAndDecrypt').mockResolvedValue({
+      decryptedText: 'password',
+      encryptionContext: { endpoint: 'http://dummy.com' },
+    });
+
+    await expect(
+      configureClient(dataSourceClientParams, clientPoolSetup, config, logger)
+    ).rejects.toThrowError(expectedErrMsg);
+
+    expect(ClientMock).toHaveBeenCalledTimes(1);
+    expect(savedObjectsMock.get).toHaveBeenCalledTimes(1);
+    expect(decodeAndDecryptSpy).toHaveBeenCalledTimes(1);
+  });
 });