Skip to content

Commit

Permalink
feat: instrument src/ with traces using OpenTelemetry
Browse files Browse the repository at this point in the history
This change implements tracing of RPC calls using
OpenTelemetry to aid in providing observability.

Fixes #2079
  • Loading branch information
odeke-em committed Aug 4, 2024
1 parent 4ec1561 commit e848ff1
Show file tree
Hide file tree
Showing 17 changed files with 2,994 additions and 1,236 deletions.
97 changes: 97 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Google APIs Client Libraries, in [Client Libraries Explained][explained].
* [Before you begin](#before-you-begin)
* [Installing the client library](#installing-the-client-library)
* [Using the client library](#using-the-client-library)
* [Observability](#observability)
* [Samples](#samples)
* [Versioning](#versioning)
* [Contributing](#contributing)
Expand Down Expand Up @@ -82,6 +83,101 @@ rows.forEach(row => console.log(row));
```


## Observability

This package has been instrumented with [OpenTelemetry](https://opentelemetry.io/docs/languages/js/) for tracing. Make sure to firstly import and enable
OpenTelemetry before importing this Spanner library.

Please use a tracer named "nodejs-spanner".

> :warning: **Make sure that the OpenTelemetry imports are the first, before importing the Spanner library**
> :warning: **In order for your spans to be annotated with SQL, you MUST opt-in by setting environment variable
`SPANNER_NODEJS_ANNOTATE_PII_SQL=1`, this is because SQL statements can be
sensitive personally-identifiable-information (PII).**

To test out trace examination, you can use Google Cloud Trace like this.

```javascript
function exportSpans(instanceId, databaseId, projectId) {
// Firstly initiate OpenTelemetry
const {Resource} = require('@opentelemetry/resources');
const {NodeSDK} = require('@opentelemetry/sdk-node');
const {trace} = require('@opentelemetry/api');
const {
NodeTracerProvider,
TraceIdRatioBasedSampler,
} = require('@opentelemetry/sdk-trace-node');
const {BatchSpanProcessor} = require('@opentelemetry/sdk-trace-base');
const {
SEMRESATTRS_SERVICE_NAME,
SEMRESATTRS_SERVICE_VERSION,
} = require('@opentelemetry/semantic-conventions');

const resource = Resource.default().merge(
new Resource({
[SEMRESATTRS_SERVICE_NAME]: 'spanner-sample',
[SEMRESATTRS_SERVICE_VERSION]: 'v1.0.0', // Whatever version of your app is running.,
})
);

const {TraceExporter} = require('@google-cloud/opentelemetry-cloud-trace-exporter');
const exporter = new TraceExporter({});

const sdk = new NodeSDK({
resource: resource,
traceExporter: exporter,
// Trace every single request to ensure that we generate
// enough traffic for proper examination of traces.
sampler: new TraceIdRatioBasedSampler(1.0),
});
sdk.start();

const provider = new NodeTracerProvider({resource: resource});
provider.addSpanProcessor(new BatchSpanProcessor(exporter));
provider.register();

// OpenTelemetry MUST be imported much earlier than the cloud-spanner package.
const tracer = trace.getTracer('nodejs-spanner');

const {Spanner} = require('@google-cloud/spanner');

tracer.startActiveSpan('deleteAndCreateDatabase', span => {
// Creates a client
const spanner = new Spanner({
projectId: projectId,
});

// Gets a reference to a Cloud Spanner instance and database
const instance = spanner.instance(instanceId);
const database = instance.database(databaseId);
const databaseAdminClient = spanner.getDatabaseAdminClient();

const databasePath = databaseAdminClient.databasePath(
projectId,
instanceId,
databaseId
);

deleteDatabase(databaseAdminClient, databasePath, () => {
createDatabase(
databaseAdminClient,
projectId,
instanceId,
databaseId,
() => {
span.end();
console.log('main span.end');
setTimeout(() => {
console.log('finished delete and creation of the database');
}, 5000);
}
);
});
});
}
```


## Samples

Expand All @@ -90,6 +186,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-spanner/tre
| Sample | Source Code | Try it |
| --------------------------- | --------------------------------- | ------ |
| Add and drop new database role | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/add-and-drop-new-database-role.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/add-and-drop-new-database-role.js,samples/README.md) |
| Export traces & observability from this library | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/observability-traces.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/observability-traces.js,samples/README.md) |
| Backups-cancel | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-cancel.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-cancel.js,samples/README.md) |
| Copies a source backup | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-copy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-copy.js,samples/README.md) |
| Backups-create-with-encryption-key | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-create-with-encryption-key.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-create-with-encryption-key.js,samples/README.md) |
Expand Down
103 changes: 103 additions & 0 deletions observability-test/grpc-instrumentation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

describe('Enabled gRPC instrumentation with sampling on', () => {
const assert = require('assert');
const {registerInstrumentations} = require('@opentelemetry/instrumentation');
const {SimpleSpanProcessor} = require('@opentelemetry/sdk-trace-base');
const {
InMemorySpanExporter,
NodeTracerProvider,
} = require('@opentelemetry/sdk-trace-node');
const {GrpcInstrumentation} = require('@opentelemetry/instrumentation-grpc');
const done = registerInstrumentations({
instrumentations: [new GrpcInstrumentation()],
});

const projectId = process.env.SPANNER_TEST_PROJECTID || 'test-project';
const {Spanner} = require('../src');
const spanner = new Spanner({
projectId: projectId,
});
const instance = spanner.instance('test-instance');
const database = instance.database('test-db');

const exporter = new InMemorySpanExporter();
const provider = new NodeTracerProvider();
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();

beforeEach(async () => {
// Mimick usual customer usage in which at setup time, the
// Spanner and Database handles are created once then sit
// and wait until they service HTTP or gRPC calls that
// come in say 5+ seconds after the service is fully started.
// This gives time for the batch session creation to be be completed.
await new Promise((resolve, reject) => setTimeout(resolve, 100));

exporter.reset();
});

after(async () => {
database.close();
spanner.close();
await provider.shutdown();
done();
});

it('Invoking database methods creates spans: gRPC enabled', async () => {
const query = {sql: 'SELECT * FROM INFORMATION_SCHEMA.TABLES'};
const [rows] = await database.run(query);
assert.ok(rows.length > 1);

// We need to ensure that spans were generated and exported.
const spans = exporter.getFinishedSpans();
assert.ok(spans.length > 0, 'at least 1 span must have been created');

// Sort the spans by duration, in the natural
// trace view order by longer duration first.
spans.sort((spanA, spanB) => {
return spanA.duration > spanB.duration;
});

const got: string[] = [];
spans.forEach(span => {
got.push(span.name);
});

const want = ['grpc.google.spanner.v1.Spanner/ExecuteStreamingSql'];

assert.deepEqual(
want,
got,
'The spans order by duration has been violated:\n\tGot: ' +
got.toString() +
'\n\tWant: ' +
want.toString()
);

// Ensure that each span has the attribute
// SEMATTRS_DB_SYSTEM, set to 'spanner'
spans.forEach(span => {
if (span.name.startsWith('cloud.google.com')) {
assert.equal(
span.attributes[SEMATTRS_DB_SYSTEM],
'spanner',
'Invalid DB_SYSTEM attribute'
);
}
});
});
});
Loading

0 comments on commit e848ff1

Please sign in to comment.