Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Capture built in metrics from System.Diagnostics.Metrics API #3052

Merged
merged 40 commits into from
Jan 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
95d6484
Added SystemDiagnosticsMetricsIntegration
jamescrosswell Jan 16, 2024
1887b87
Fixed verify tests
jamescrosswell Jan 16, 2024
345b3bd
Replaced double lock with a singleton
jamescrosswell Jan 16, 2024
1b77a91
Replaced singleton with Lazy<T>
jamescrosswell Jan 16, 2024
4686e06
Demonstrate capturing built in metrics in the Sample application
jamescrosswell Jan 17, 2024
e4e6c78
Refactoring + Changelog
jamescrosswell Jan 17, 2024
b4d9036
Update Sentry.EntityFramework.csproj
jamescrosswell Jan 17, 2024
2a1d608
Updated verified files
jamescrosswell Jan 17, 2024
a51bb02
Update build.yml
jamescrosswell Jan 17, 2024
237b6cf
Create SentryOptionsTests.Integrations_default_ones_are_properly_regi…
jamescrosswell Jan 17, 2024
17e8c92
Update build.yml
jamescrosswell Jan 17, 2024
24d102a
Merge branch 'default-metrics' of github.com:getsentry/sentry-dotnet …
jamescrosswell Jan 17, 2024
06da2b8
Merge branch 'main' into default-metrics
jamescrosswell Jan 17, 2024
c91840b
Update SubstringOrRegexPatternTests.cs
jamescrosswell Jan 17, 2024
40eef42
Update SentryOptionsExtensionsTests.cs
jamescrosswell Jan 17, 2024
fb561cf
Some basic SystemDiagnosticsMetricsListenerTests
jamescrosswell Jan 17, 2024
421a4d8
SystemDiagnosticsMetricsIntegrationTests
jamescrosswell Jan 18, 2024
72c744b
Added tests to check aggregates
jamescrosswell Jan 18, 2024
9ecdbe7
Added support for UpDownCounters
jamescrosswell Jan 18, 2024
8e22090
Fixed tests to avoid metric collisions between different tests
jamescrosswell Jan 18, 2024
a27d50c
Delay resolution of MetricAggregator until SentryInit has completed (…
jamescrosswell Jan 18, 2024
d987528
Renamed SetWithConfigBinding to WIthConfigBinding
jamescrosswell Jan 18, 2024
bb9b269
Convert to async
jamescrosswell Jan 18, 2024
79352dc
Fixed typo
jamescrosswell Jan 19, 2024
ebf8ddb
Added ability to configure Meters to listen to (in addition to specif…
jamescrosswell Jan 19, 2024
7bdc9a4
Update ApiApprovalTests.Run.Net4_8.verified.txt
jamescrosswell Jan 19, 2024
d23afaf
Programmatically ensure unique names for test meters/instruments
jamescrosswell Jan 19, 2024
4bc84b1
Merge branch 'main' into default-metrics
jamescrosswell Jan 22, 2024
383d0c7
Merge branch 'main' into default-metrics
jamescrosswell Jan 22, 2024
e2bad12
Update SentryOptionsTests.verify.cs
jamescrosswell Jan 22, 2024
a9a6c93
Updated verify tests
jamescrosswell Jan 22, 2024
3baaa0b
Updated verify tests
jamescrosswell Jan 23, 2024
c1ac0fc
Update SentryOptionsTests.verify.cs
jamescrosswell Jan 23, 2024
77e6880
Added Windows specific SentryOptions verify tests
jamescrosswell Jan 23, 2024
33d8af9
Collect all built in metrics by default
jamescrosswell Jan 23, 2024
4f84c9d
Refactoring
jamescrosswell Jan 23, 2024
93b7afb
Update SystemDiagnosticsMetricsIntegrationTests.cs
jamescrosswell Jan 23, 2024
c358a60
Merge branch 'main' into default-metrics
jamescrosswell Jan 30, 2024
a7a8e1f
Update Program.cs
jamescrosswell Jan 30, 2024
e26175e
Integrating review feedback
jamescrosswell Jan 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### Features

- Added support for capturing built in metrics from the System.Diagnostics.Metrics API ([#3052](https://github.com/getsentry/sentry-dotnet/pull/3052))

### Significant change in behavior

- Added `Sentry` namespace to global usings when `ImplicitUsings` is enabled ([#3043](https://github.com/getsentry/sentry-dotnet/pull/3043))
Expand Down
84 changes: 60 additions & 24 deletions samples/Sentry.Samples.Console.Metrics/Program.cs
Original file line number Diff line number Diff line change
@@ -1,10 +1,20 @@
namespace Sentry.Samples.Console.Metrics;
using System.Diagnostics.Metrics;
using System.Text.RegularExpressions;

namespace Sentry.Samples.Console.Metrics;

internal static class Program
{
private static readonly Random Roll = new();
private static readonly Random Roll = Random.Shared;

// Sentry also supports capturing System.Diagnostics.Metrics
private static readonly Meter HatsMeter = new("HatCo.HatStore", "1.0.0");
private static readonly Counter<int> HatsSold = HatsMeter.CreateCounter<int>(
name: "hats-sold",
unit: "Hats",
description: "The number of hats sold in our store");

private static void Main()
private static async Task Main()
{
// Enable the SDK
using (SentrySdk.Init(options =>
Expand All @@ -20,41 +30,44 @@ private static void Main()
// Initialize some (non null) ExperimentalMetricsOptions to enable Sentry Metrics,
options.ExperimentalMetrics = new ExperimentalMetricsOptions
{
EnableCodeLocations =
true // Set this to false if you don't want to track code locations for some reason
EnableCodeLocations = true, // Set this to false if you don't want to track code locations
CaptureSystemDiagnosticsInstruments = [
// Capture System.Diagnostics.Metrics matching the name "HatCo.HatStore", which is the name
// of the custom HatsMeter defined above
"hats-sold"
],
// Capture all built in metrics (this is the default - you can override this to capture some or
// none of these if you prefer)
CaptureSystemDiagnosticsMeters = BuiltInSystemDiagnosticsMeters.All
};
}))
{
System.Console.WriteLine("Measure, Yeah, Measure!");
Action[] actions =
[
() => PlaySetBingo(10),
() => CreateRevenueGauge(100),
() => MeasureShrimp(30),
];
while (true)

Action[] actions = [PlaySetBingo, CreateRevenueGauge, MeasureShrimp, SellHats];
do
{
// Perform your task here
var actionIdx = Roll.Next(0, actions.Length);
actions[actionIdx]();
// Run a random action
var idx = Roll.Next(0, actions.Length);
actions[idx]();

// Make an API call
await CallSampleApiAsync();

// Optional: Delay to prevent tight looping
var sleepTime = Roll.Next(1, 10);
var sleepTime = Roll.Next(1, 5);
System.Console.WriteLine($"Sleeping for {sleepTime} second(s).");
System.Console.WriteLine("Press any key to stop...");
Thread.Sleep(TimeSpan.FromSeconds(sleepTime));
// Check if a key has been pressed
if (System.Console.KeyAvailable)
{
break;
}
}
while (!System.Console.KeyAvailable);
System.Console.WriteLine("Measure up");
}
}

private static void PlaySetBingo(int attempts)
private static void PlaySetBingo()
{
const int attempts = 10;
var solution = new[] { 3, 5, 7, 11, 13, 17 };

// StartTimer creates a distribution that is designed to measure the amount of time it takes to run code
Expand All @@ -74,8 +87,9 @@ private static void PlaySetBingo(int attempts)
}
}

private static void CreateRevenueGauge(int sampleCount)
private static void CreateRevenueGauge()
{
const int sampleCount = 100;
using (SentrySdk.Metrics.StartTimer(nameof(CreateRevenueGauge), MeasurementUnit.Duration.Millisecond))
{
for (var i = 0; i < sampleCount; i++)
Expand All @@ -88,8 +102,9 @@ private static void CreateRevenueGauge(int sampleCount)
}
}

private static void MeasureShrimp(int sampleCount)
private static void MeasureShrimp()
{
const int sampleCount = 30;
using (SentrySdk.Metrics.StartTimer(nameof(MeasureShrimp), MeasurementUnit.Duration.Millisecond))
{
for (var i = 0; i < sampleCount; i++)
Expand All @@ -100,4 +115,25 @@ private static void MeasureShrimp(int sampleCount)
}
}
}

private static void SellHats()
{
// Here we're emitting the metric using System.Diagnostics.Metrics instead of SentrySdk.Metrics.
// We won't see accurate code locations for these, so Sentry.Metrics are preferable but support
// for System.Diagnostics.Metrics means Sentry can collect a bunch built in metrics without you
// having to instrument anything... see case 4 below
HatsSold.Add(Roll.Next(0, 1000));
}

private static async Task CallSampleApiAsync()
{
// Here we demonstrate collecting some built in metrics for HTTP requests... this works because
// we've configured ExperimentalMetricsOptions.CaptureInstruments to match "http.client.*"
//
// See https://learn.microsoft.com/en-us/dotnet/core/diagnostics/built-in-metrics-system-net#systemnethttp
var httpClient = new HttpClient();
var url = "https://api.sampleapis.com/coffee/hot";
var result = await httpClient.GetAsync(url);
System.Console.WriteLine($"GET {url} {result.StatusCode}");
}
}
173 changes: 173 additions & 0 deletions src/Sentry/BuiltInSystemDiagnosticsMeters.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
namespace Sentry;

/// <summary>
/// Well known values for built in metrics that can be configured for
/// <see cref="ExperimentalMetricsOptions.CaptureSystemDiagnosticsMeters"/>
/// </summary>
public static partial class BuiltInSystemDiagnosticsMeters
{
private const string MicrosoftAspNetCoreHostingPattern = @"^Microsoft\.AspNetCore\.Hosting$";
private const string MicrosoftAspNetCoreRoutingPattern = @"^Microsoft\.AspNetCore\.Routing$";
private const string MicrosoftAspNetCoreDiagnosticsPattern = @"^Microsoft\.AspNetCore\.Diagnostics$";
private const string MicrosoftAspNetCoreRateLimitingPattern = @"^Microsoft\.AspNetCore\.RateLimiting$";
private const string MicrosoftAspNetCoreHeaderParsingPattern = @"^Microsoft\.AspNetCore\.HeaderParsing$";
private const string MicrosoftAspNetCoreServerKestrelPattern = @"^Microsoft\.AspNetCore\.Server\.Kestrel$";
private const string MicrosoftAspNetCoreHttpConnectionsPattern = @"^Microsoft\.AspNetCore\.Http\.Connections$";
private const string MicrosoftExtensionsDiagnosticsHealthChecksPattern = @"^Microsoft\.Extensions\.Diagnostics\.HealthChecks$";
private const string MicrosoftExtensionsDiagnosticsResourceMonitoringPattern = @"^Microsoft\.Extensions\.Diagnostics\.ResourceMonitoring$";
private const string SystemNetNameResolutionPattern = @"^System\.Net\.NameResolution$";
private const string SystemNetHttpPattern = @"^System\.Net\.Http$";

/// <summary>
/// Matches the built in Microsoft.AspNetCore.Hosting metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreHosting = MicrosoftAspNetCoreHostingRegex();

[GeneratedRegex(MicrosoftAspNetCoreHostingPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftAspNetCoreHostingRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreHosting = new Regex(MicrosoftAspNetCoreHostingPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.AspNetCore.Routing metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreRouting = MicrosoftAspNetCoreRoutingRegex();

[GeneratedRegex(MicrosoftAspNetCoreRoutingPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftAspNetCoreRoutingRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreRouting = new Regex(MicrosoftAspNetCoreRoutingPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.AspNetCore.Diagnostics metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreDiagnostics = MicrosoftAspNetCoreDiagnosticsRegex();

[GeneratedRegex(MicrosoftAspNetCoreDiagnosticsPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftAspNetCoreDiagnosticsRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreDiagnostics = new Regex(MicrosoftAspNetCoreDiagnosticsPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.AspNetCore.RateLimiting metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreRateLimiting = MicrosoftAspNetCoreRateLimitingRegex();

[GeneratedRegex(MicrosoftAspNetCoreRateLimitingPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftAspNetCoreRateLimitingRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreRateLimiting = new Regex(MicrosoftAspNetCoreRateLimitingPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.AspNetCore.HeaderParsing metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreHeaderParsing = MicrosoftAspNetCoreHeaderParsingRegex();

[GeneratedRegex(MicrosoftAspNetCoreHeaderParsingPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftAspNetCoreHeaderParsingRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreHeaderParsing = new Regex(MicrosoftAspNetCoreHeaderParsingPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.AspNetCore.Server.Kestrel metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreServerKestrel = MicrosoftAspNetCoreServerKestrelRegex();

[GeneratedRegex(MicrosoftAspNetCoreServerKestrelPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftAspNetCoreServerKestrelRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreServerKestrel = new Regex(MicrosoftAspNetCoreServerKestrelPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.AspNetCore.Http.Connections metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreHttpConnections = MicrosoftAspNetCoreHttpConnectionsRegex();

[GeneratedRegex(MicrosoftAspNetCoreHttpConnectionsPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftAspNetCoreHttpConnectionsRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftAspNetCoreHttpConnections = new Regex(MicrosoftAspNetCoreHttpConnectionsPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.Extensions.Diagnostics.HealthChecks metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftExtensionsDiagnosticsHealthChecks = MicrosoftExtensionsDiagnosticsHealthChecksRegex();

[GeneratedRegex(MicrosoftExtensionsDiagnosticsHealthChecksPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftExtensionsDiagnosticsHealthChecksRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftExtensionsDiagnosticsHealthChecks = new Regex(MicrosoftExtensionsDiagnosticsHealthChecksPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in Microsoft.Extensions.Diagnostics.ResourceMonitoring metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern MicrosoftExtensionsDiagnosticsResourceMonitoring = MicrosoftExtensionsDiagnosticsResourceMonitoringRegex();

[GeneratedRegex(MicrosoftExtensionsDiagnosticsResourceMonitoringPattern, RegexOptions.Compiled)]
private static partial Regex MicrosoftExtensionsDiagnosticsResourceMonitoringRegex();
#else
public static readonly SubstringOrRegexPattern MicrosoftExtensionsDiagnosticsResourceMonitoring = new Regex(MicrosoftExtensionsDiagnosticsResourceMonitoringPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in System.Net.NameResolution metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern SystemNetNameResolution = SystemNetNameResolutionRegex();

[GeneratedRegex(SystemNetNameResolutionPattern, RegexOptions.Compiled)]
private static partial Regex SystemNetNameResolutionRegex();
#else
public static readonly SubstringOrRegexPattern SystemNetNameResolution = new Regex(SystemNetNameResolutionPattern, RegexOptions.Compiled);
#endif

/// <summary>
/// Matches the built in <see cref="System.Net.Http"/> metrics
/// </summary>
#if NET8_0_OR_GREATER
public static readonly SubstringOrRegexPattern SystemNetHttp = SystemNetHttpRegex();

[GeneratedRegex(SystemNetHttpPattern, RegexOptions.Compiled)]
private static partial Regex SystemNetHttpRegex();
#else
public static readonly SubstringOrRegexPattern SystemNetHttp = new Regex(SystemNetHttpPattern, RegexOptions.Compiled);
#endif

private static readonly Lazy<IList<SubstringOrRegexPattern>> LazyAll = new(() => new List<SubstringOrRegexPattern>
{
MicrosoftAspNetCoreHosting,
MicrosoftAspNetCoreRouting,
MicrosoftAspNetCoreDiagnostics,
MicrosoftAspNetCoreRateLimiting,
MicrosoftAspNetCoreHeaderParsing,
MicrosoftAspNetCoreServerKestrel,
MicrosoftAspNetCoreHttpConnections,
SystemNetNameResolution,
SystemNetHttp,
MicrosoftExtensionsDiagnosticsHealthChecks,
MicrosoftExtensionsDiagnosticsResourceMonitoring
});

/// <summary>
/// Matches all built in metrics
/// </summary>
/// <returns></returns>
public static IList<SubstringOrRegexPattern> All => LazyAll.Value;
}
59 changes: 59 additions & 0 deletions src/Sentry/ExperimentalMetricsOptions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
namespace Sentry;

/// <summary>
/// Settings for the experimental Metrics feature. This feature is preview only and will very likely change in the future
/// without a major version bump... so use at your own risk.
/// </summary>
public class ExperimentalMetricsOptions
{
/// <summary>
/// Determines whether code locations should be recorded for Metrics
/// </summary>
public bool EnableCodeLocations { get; set; } = true;

private IList<SubstringOrRegexPattern> _captureSystemDiagnosticsInstruments = new List<SubstringOrRegexPattern>();

/// <summary>
/// <para>
/// A list of Substrings or Regular Expressions. Any `System.Diagnostics.Metrics.Instrument` whose name
/// matches one of the items in this list will be collected and reported to Sentry.
/// </para>
/// <para>
/// These can be either custom Instruments that you have created or any of the built in metrics that are available.
/// </para>
/// <para>
/// See https://learn.microsoft.com/en-us/dotnet/core/diagnostics/built-in-metrics for more information.
/// </para>
/// </summary>
public IList<SubstringOrRegexPattern> CaptureSystemDiagnosticsInstruments
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In terms of defaults, we could capture none/some/all of these build in metrics.

Are there quotas that we need to be mindful of (so are we potentially going to chew through someone's entire Sentry Quota if we start collecting routing match attempt metrics?

If not, all of those built in metrics are potentially useful/interesting to people so we could enable these.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this PR to include all of the built in metrics by default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is experimental I think it's fine to include it by default but definitely worth keeping an eye on impact.
Cost is usually driven by cardinality so if the metrics have no tags it could be OK. cc @bitsandfoxes to own this conversation internally

Copy link
Collaborator Author

@jamescrosswell jamescrosswell Jan 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metrics definitely have tags. For example, http.server.request.duration has 7 attributes, which add dimensionality represented as tags in Sentry. In most situations, that will result in 1 different metric per route configured in the ASP.NET Core application... but that's just one example.

Basically anything that's documented as an Attribute in the built in metrics documentation turns up as a tag in Sentry.

Maybe we don't need some of the built in metrics as the insights are already available via our Tracing instrumentation for many of them?

{
// NOTE: During configuration binding, .NET 6 and lower used to just call Add on the existing item.
// .NET 7 changed this to call the setter with an array that already starts with the old value.
// We have to handle both cases.
get => _captureSystemDiagnosticsInstruments;
set => _captureSystemDiagnosticsInstruments = value.WithConfigBinding();
}

private IList<SubstringOrRegexPattern> _captureSystemDiagnosticsMeters = BuiltInSystemDiagnosticsMeters.All;

/// <summary>
/// <para>
/// A list of Substrings or Regular Expressions. Instruments for any `System.Diagnostics.Metrics.Meter`
/// whose name matches one of the items in this list will be collected and reported to Sentry.
/// </para>
/// <para>
/// These can be either custom Instruments that you have created or any of the built in metrics that are available.
/// </para>
/// <para>
/// See https://learn.microsoft.com/en-us/dotnet/core/diagnostics/built-in-metrics for more information.
/// </para>
/// </summary>
public IList<SubstringOrRegexPattern> CaptureSystemDiagnosticsMeters
{
// NOTE: During configuration binding, .NET 6 and lower used to just call Add on the existing item.
// .NET 7 changed this to call the setter with an array that already starts with the old value.
// We have to handle both cases.
get => _captureSystemDiagnosticsMeters;
set => _captureSystemDiagnosticsMeters = value.WithConfigBinding();
}
}
Loading
Loading