[Bug]: creating multiple gauges causes data to be copied from one to the other and sent to collector

[bouk]

What happened?

Hi,

I'm creating multiple observable gauges and I'm seeing that if I don't set gauges myself, old values of other gauges get copied into it somehow. I'm emitting metrics to the opentelemetry collector.

Reproduction

use opentelemetry::global;
use opentelemetry_sdk::runtime::Tokio;
use std::time::Duration;

#[tokio::main]
async fn main() -> eyre::Result<()> {
    opentelemetry_otlp::new_pipeline()
        .metrics(Tokio)
        .with_period(Duration::from_secs(10))
        .with_exporter(opentelemetry_otlp::new_exporter().tonic())
        .build()?;

    let meter = global::meter("test");
    let a = meter.i64_observable_gauge("a").init();
    let b = meter.i64_observable_gauge("b").init();

    meter.register_callback(&[b.as_any()], move |observer| {
        observer.observe_i64(&b, 123, &[]);
    })?;

    let mut interval = tokio::time::interval(Duration::from_secs(60));
    loop {
        interval.tick().await;
    }
}

Logs note

I configured opentelemetry-collector with the debug exporter with verbosity = "detailed". You can see that the first export of metrics is fine, and then the second contains an extra entry for the 'a' gauge even though it doesn't get observed anywhere. The value is the value of the b gauge.

API Version

0.21

SDK Version

0.21

What Exporters are you seeing the problem on?

No response

Relevant log output

2023-11-09T17:43:39.616+0100        info        MetricsExporter        {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2023-11-09T17:43:39.616+0100        info        ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.6.1
Resource attributes:
     -> service.name: Str(unknown_service)
     -> telemetry.sdk.version: Str(0.21.0)
     -> telemetry.sdk.language: Str(rust)
     -> telemetry.sdk.name: Str(opentelemetry)
     -> host.name: Str(test)
     -> os.type: Str(linux)
ScopeMetrics #0
ScopeMetrics SchemaURL:
InstrumentationScope test
Metric #0
Descriptor:
     -> Name: b
     -> Description:
     -> Unit:
     -> DataType: Gauge
NumberDataPoints #0
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2023-11-09 16:43:39.615632922 +0000 UTC
Value: 123
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}

... some time passes ...

2023-11-09T17:43:49.615+0100        info        ResourceMetrics #0
2023-11-09T17:43:49.615+0100        info        MetricsExporter        {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 2, "data points": 2}
Resource SchemaURL: https://opentelemetry.io/schemas/1.6.1
Resource attributes:
     -> service.name: Str(unknown_service)
     -> telemetry.sdk.version: Str(0.21.0)
     -> telemetry.sdk.language: Str(rust)
     -> telemetry.sdk.name: Str(opentelemetry)
     -> host.name: Str(test)
     -> os.type: Str(linux)
ScopeMetrics #0
ScopeMetrics SchemaURL:
InstrumentationScope test
Metric #0
Descriptor:
     -> Name: a
     -> Description:
     -> Unit:
     -> DataType: Gauge
NumberDataPoints #0
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2023-11-09 16:43:39.615632922 +0000 UTC
Value: 123
Metric #1
Descriptor:
     -> Name: b
     -> Description:
     -> Unit:
     -> DataType: Gauge
NumberDataPoints #0
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2023-11-09 16:43:49.615136702 +0000 UTC
Value: 123
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}

[jtescher]

@bouk thanks for reporting this. It does appear to be a regression when gauges are not written to (last value aggregation without a last value). Should have a fix shortly.