Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Monotonic Types #145

Merged
merged 4 commits into from
Jun 2, 2020
Merged

Conversation

MrAlias
Copy link
Contributor

@MrAlias MrAlias commented Apr 28, 2020

Add the following types:

  • MONOTONIC_INT64
  • MONOTONIC_DOUBLE

These type values are still transported in the same DataPoints as their non-monotonic counterpoints, but add the significance to their values as being monotonic.

This is a partial replacement of #137
Relates to #125

Knowing the monotonicity of metric data enables receivers to implement
certain algorithms to calculate the rate of change of the data. As such,
this is an important quality to describe and is added to the
`MetricDescriptor`.

*Note on English*: NONDECREASING is used to describe values that either
increase or remain constant over time. The English term "increasing" is
not used as it does not encompass values that remain constant over time.

A comment is added to identify the omission of NONINCREASING and
information on how to notify it is needed should that arise in the
future.
@tigrannajaryan
Copy link
Member

With 3 different enums we now use 3x32=96bits to encode 6 bits worth of info. This can have significant impact on memory consumption in Collector which may hold many thousands or even millions of metrics in queues.

Should we use a single field with bitmasks instead? We can keep the enums and just use a single uint32 with lots of reserved fields for future use.

@bogdandrutu
Copy link
Member

@tigrannajaryan I see that as an optimization, which I think we can do it as a next step after we finalize the data format and all the properties we need.

@tigrannajaryan
Copy link
Member

@MrAlias @bogdandrutu can we move forward with this? We are waiting for the finalization of metrics ProtoBufs to be able to use them in the Collector.

@MrAlias
Copy link
Contributor Author

MrAlias commented May 28, 2020

@MrAlias @bogdandrutu can we move forward with this? We are waiting for the finalization of metrics ProtoBufs to be able to use them in the Collector.

Sorry, was blocked working on open-telemetry/opentelemetry-collector-contrib#229 for a few days, but will get back to this today.

Talking with @bogdandrutu last week it sounds like two things (related to this PR) should be done:

  • The protobuf should describe data post aggregation (instead of pre-instrument). This means the blocking conversation could be resolved by keeping things as is, but ...
  • The concept of Monotonic doesn't apply to all datapoints (SummaryDataPoints), therefore, we should move the concept directly to the Type itself.

Going to update later to remove the Monotonic field and change Type to:

// Type is the type of values a metric has.
enum Type {
  INVALID_TYPE = 0;

  INT64 = 1;
  MONOTONIC_INT64 = 2;

  DOUBLE = 3;
  MONOTONIC_DOUBLE = 4;

  HISTOGRAM = 5;
  SUMMARY = 6;
}

There was an idea to include a MONOTONIC_HISTOGRAM as well, but I'm not sure how useful it will be. I'm open to opinions on this.

@MrAlias
Copy link
Contributor Author

MrAlias commented May 28, 2020

Looks like @jmacd has also been thinking about this: #125 (comment). Going to wait until after the Metric SIG meeting today to make any changes to make sure we are all in sync.

@jmacd
Copy link
Contributor

jmacd commented May 28, 2020

I think we found resolution today in the SIG call.

@bogdandrutu
Copy link
Member

@jmacd can you document the proposed resolution?

@MrAlias
Copy link
Contributor Author

MrAlias commented May 29, 2020

@jmacd can you document the proposed resolution?

@bogdandrutu : The plan is to go with:

// Type is the type of values a metric has.
enum Type {
  INVALID_TYPE = 0;

  INT64 = 1;
  MONOTONIC_INT64 = 2;

  DOUBLE = 3;
  MONOTONIC_DOUBLE = 4;

  HISTOGRAM = 5;
  SUMMARY = 6;
}

thoughts?

Instead of including a monotonic refinement, specify where applicable to
the type.
@bogdandrutu
Copy link
Member

Can you please update the description?

@MrAlias MrAlias changed the title Add Monotonic refinement Add Monotonic DataPoints Jun 1, 2020
@MrAlias MrAlias changed the title Add Monotonic DataPoints Add Monotonic Types Jun 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants