From b5bcddc84f992265a4cf3f93f2a78301ad5967ad Mon Sep 17 00:00:00 2001 From: Noah Falk Date: Wed, 17 Mar 2021 03:07:32 -0700 Subject: [PATCH] Applying review feedback --- ...ributed-tracing-collection-walkthroughs.md | 151 +++++++++++------- .../distributed-tracing-concepts.md | 112 ++++++------- ...ed-tracing-instrumentation-walkthroughs.md | 92 +++++++---- docs/core/diagnostics/distributed-tracing.md | 32 ++-- 4 files changed, 227 insertions(+), 160 deletions(-) diff --git a/docs/core/diagnostics/distributed-tracing-collection-walkthroughs.md b/docs/core/diagnostics/distributed-tracing-collection-walkthroughs.md index 649dfbeeebe8d..5d13334d5f233 100644 --- a/docs/core/diagnostics/distributed-tracing-collection-walkthroughs.md +++ b/docs/core/diagnostics/distributed-tracing-collection-walkthroughs.md @@ -1,21 +1,21 @@ --- title: Collect a distributed trace - .NET -description: A tutorial to collect distributed traces in .NET applications +description: Tutorials to collect distributed traces in .NET applications using OpenTelemetry, Application Insights, or ActivityListener ms.topic: tutorial ms.date: 03/14/2021 --- # Collect a distributed trace -**This article applies to: ✔️** .NET Core 5.0 and later versions **or** any .NET application using the -[DiagnosticSource NuGet package](https://www.nuget.org/packages/System.Diagnostics.DiagnosticSource/5.0.1) version 5 or later +**This article applies to: ✔️** .NET Core 2.1 and later versions **and** .NET Framework 4.5 and later versions -.NET applications can be instrumented using the API to produce -distributed tracing telemetry. In this tutorial you will record this instrumented telemetry with different telemetry collection -libraries so that it is available to diagnose application issues. See +Instrumented code can create objects as part of a distributed trace, but the information +in these objects needs to be collected into a centralized persistant store so that the entire trace can be +reviewed later. In this tutorial you will collect the distributed trace telemetry in different ways so that it is +available to diagnose application issues when needed. See [the instrumentation tutorial](distributed-tracing-instrumentation-walkthroughs.md) if you need to add new instrumentation. -## Collect using OpenTelemetry +## Collect traces using OpenTelemetry ### Prerequisites @@ -31,16 +31,17 @@ dotnet new console Applications that target .NET 5 and later already have the necessary distributed tracing APIs included. For apps targeting older .NET versions add the [System.Diagnostics.DiagnosticSource NuGet package](https://www.nuget.org/packages/System.Diagnostics.DiagnosticSource/) -version 5.0.1 or greater. +version 5 or greater. + ```dotnetcli -dotnet add package System.Diagnostics.DiagnosticSource --version 5.0.1 +dotnet add package System.Diagnostics.DiagnosticSource ``` Replace the contents of the generated Program.cs with this example source: + ```C# using System; using System.Diagnostics; -using System.Net.Http; using System.Threading.Tasks; namespace Sample.DistributedTracing @@ -83,15 +84,23 @@ namespace Sample.DistributedTracing } ``` -Running the app does not log anything yet +Running the app does not record any tracing information yet + ```dotnetcli > dotnet run Example work done ``` -### Add logging using OpenTelemetry +### Collect using OpenTelemetry -Add the [OpenTelemetry](https://www.nuget.org/packages/OpenTelemetry/) and +[OpenTelemetry](https://opentelemetry.io/) is a vendor neutral open source project supported by the +[Cloud Native Computing Foundation](https://www.cncf.io/) that aims to standardize generating and collecting telemetry for +cloud-native software. In this example you will collect and display distributed trace information on the console though +OpenTelemetry can be reconfigured to send it elsewhere. See the +[OpenTelemetry getting started guide](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/trace/getting-started/README.md) +for more information. + +Add the [OpenTelemetry](https://www.nuget.org/packages/OpenTelemetry/) and [OpenTelemetry.Exporter.Console](https://www.nuget.org/packages/OpenTelemetry.Exporter.Console/) NuGet packages. ```dotnetcli @@ -99,7 +108,8 @@ dotnet add package OpenTelemetry dotnet add package OpenTelemetry.Exporter.Console ``` -Update Program.cs with additional using statments: +Update Program.cs with additional OpenTelemetry using statments: + ```C# using OpenTelemetry; using OpenTelemetry.Trace; @@ -108,15 +118,13 @@ using System.Diagnostics; using System.Threading.Tasks; ``` -And update Main() to create the OpenTelemetry TracerProvider: +Update Main() to create the OpenTelemetry TracerProvider: + ```C# public static async Task Main() { using var tracerProvider = Sdk.CreateTracerProviderBuilder() - .SetSampler(new AlwaysOnSampler()) - // Add more libraries .AddSource("Sample.DistributedTracing") - // Add more exporters .AddConsoleExporter() .Build(); @@ -125,7 +133,8 @@ And update Main() to create the OpenTelemetry TracerProvider: } ``` -Now the app logs distributed trace information to the console: +Now the app collects distributed trace information and displays it to the console: + ```dotnetcli > dotnet run Activity.Id: 00-35c0e68b0dac3c49be08a9d9cab32579-0b7477e11aa20d40-01 @@ -157,29 +166,45 @@ Resource associated with Activity: Example work done ``` +#### Sources + +In the example code you invoked `AddSource("Sample.DistributedTracing")` so that OpenTelemetry would +capture the Activities produced by the ActivitySource that was already present in the code: + +```csharp + static ActivitySource s_source = new ActivitySource("Sample.DistributedTracing"); +``` + +Telemetry from any ActivitySource can captured by calling AddSource() with the source's name. + +#### Exporters + The console exporter is helpful for quick examples or local development but in a production deployment -you will probably want to send logs to a central logging store. OpenTelemetry supports a variety -of different logging destinations using different +you will probably want to send traces to a centralized store. OpenTelemetry supports a variety +of destinations using different [exporters](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#exporter-library). -See the [OpenTelemetry getting started](https://github.com/open-telemetry/opentelemetry-dotnet#getting-started) -instructions for more information on configuring OpenTelemetry. +See the [OpenTelemetry getting started guide](https://github.com/open-telemetry/opentelemetry-dotnet#getting-started) +for more information on configuring OpenTelemetry. -## Collect using Application Insights +## Collect traces using Application Insights -Distributed tracing telemetry is automatically captured after configuring the Application Insights SDK -([.NET](https://docs.microsoft.com/en-us/azure/azure-monitor/app/asp-net), [.NET Core](https://docs.microsoft.com/en-us/azure/azure-monitor/app/asp-net-core)) -or by enabling [code-less instrumentation](https://docs.microsoft.com/en-us/azure/azure-monitor/app/codeless-overview). +Distributed tracing telemetry is automatically captured after configuring the Application Insights SDK +([ASP.NET](https://docs.microsoft.com/azure/azure-monitor/app/asp-net), [ASP.NET Core](https://docs.microsoft.com/azure/azure-monitor/app/asp-net-core)) +or by enabling [code-less instrumentation](https://docs.microsoft.com/azure/azure-monitor/app/codeless-overview). -See the [Application Insights distributed tracing documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/app/distributed-tracing) for more +See the [Application Insights distributed tracing documentation](https://docs.microsoft.com/azure/azure-monitor/app/distributed-tracing) for more information. > [!NOTE] > Currently Application Insights only supports collecting specific well-known Activity instrumentation and will ignore new user added Activities. Application -> Insights offers [TrackDependency](https://docs.microsoft.com/en-us/azure/azure-monitor/app/api-custom-events-metrics#trackdependency) as a vendor +> Insights offers [TrackDependency](https://docs.microsoft.com/azure/azure-monitor/app/api-custom-events-metrics#trackdependency) as a vendor > specific API for adding custom distributed tracing information. +## Collect traces using custom logic -## Collect using a custom logging implementation +Developers are free to create their own customized collection logic for Activity trace data. This example collects the +telemetry using the API provided by .NET and prints +it to the console. ### Prerequisites @@ -187,22 +212,25 @@ information. ### Create an example application +First you will create an example application that has some distributed trace instrumentation but no trace data is being collected. + ```dotnetcli dotnet new console ``` Applications that target .NET 5 and later already have the necessary distributed tracing APIs included. For apps targeting older .NET versions add the [System.Diagnostics.DiagnosticSource NuGet package](https://www.nuget.org/packages/System.Diagnostics.DiagnosticSource/) -version 5.0.1 or greater. +version 5 or greater. + ```dotnetcli -dotnet add package System.Diagnostics.DiagnosticSource --version 5.0.1 +dotnet add package System.Diagnostics.DiagnosticSource ``` Replace the contents of the generated Program.cs with this example source: + ```C# using System; using System.Diagnostics; -using System.Net.Http; using System.Threading.Tasks; namespace Sample.DistributedTracing @@ -245,19 +273,23 @@ namespace Sample.DistributedTracing } ``` -Running the app does not log anything yet +Running the app does not collect any trace data yet: + ```dotnetcli > dotnet run Example work done ``` -### Add code to log the Activities +### Add code to observe the Activities + +Update Main() with this code: -Update Main() with this code that logs Activities: ```C# static async Task Main(string[] args) { Activity.DefaultIdFormat = ActivityIdFormat.W3C; + Activity.ForceDefaultIdFormat = true; + Console.WriteLine(" {0,-15} {1,-60} {2,-15}", "OperationName", "Id", "Duration"); ActivitySource.AddActivityListener(new ActivityListener() { @@ -273,6 +305,7 @@ Update Main() with this code that logs Activities: ``` The output now includes logging: + ```dotnetcli > dotnet run OperationName Id Duration @@ -285,41 +318,41 @@ Stopped: SomeWork 00-bdb5faffc2fc1548b6ba49a31c4a0ae0-c447fb302059784f-01 Example work done ``` -Setting is optional -but helps ensure the sample produces similar output on different .NET runtime versions. .NET 5.0 uses -the W3C ID format by default but earlier .NET versions default to using - as a precaution -to avoid compatibility issues with older distributed tracing systems. See +Setting and + is optional +but helps ensure the sample produces similar output on different .NET runtime versions. .NET 5 uses +the W3C TraceContext ID format by default but earlier .NET versions default to using + ID format. See [Activity IDs](distributed-tracing-concepts.md#activity-ids) for more details. is used to receive callbacks -during the lifetime of an Activity. - - - Each +during the lifetime of an Activity. +- - Each Activity is associated with an ActivitySource which acts as a namespace for a set of Activities. This callback is invoked once for each ActivitySource in the process. Returning true indicates the listener should be notified about Activities associated with this source. - - - By default - does not +- - By default + does not create an Activity object unless some ActivityListener indicates it should be sampled. Returning - + indicates that the Activity should be created, - should be set -to true, and -will have the + should be set +to true, and +will have the flag set. IsAllDataRequested can be observed by the instrumented code as a hint that a listener wants to ensure that auxilliary Activity information such as Tags and Events are populated. -The Recorded flag is encoded in the W3C ID and is a hint to other processes involved in the -distributed trace that this trace should be logged. - - and - are +The Recorded flag is encoded in the W3C TraceContext ID and is a hint to other processes +involved in the distributed trace that this trace should be sampled. + - and + are called when an Activity is started and stopped respectively. These callbacks provide an -oportunity to log any relevant information about the Activity. When an Activity has just -started much of the data may still be incomplete and it will be filled in before the Activity -stops. +oportunity to record relevant information about the Activity or potentially to modify it. +When an Activity has just started much of the data may still be incomplete and it will +be populated before the Activity stops. -Once an ActivityListener has been created and the callbacks are populated, invoking - -initiates invoking the callbacks. Call +Once an ActivityListener has been created and the callbacks are populated, calling + +initiates invoking the callbacks. Call to stop the flow of callbacks. Beware that in multi-threaded code callback notifications in progress could be received while Dispose() is running or even very shortly after it has diff --git a/docs/core/diagnostics/distributed-tracing-concepts.md b/docs/core/diagnostics/distributed-tracing-concepts.md index 64141672f1b81..3f849905c8eb5 100644 --- a/docs/core/diagnostics/distributed-tracing-concepts.md +++ b/docs/core/diagnostics/distributed-tracing-concepts.md @@ -4,19 +4,17 @@ description: .NET distributed tracing concepts ms.date: 03/14/2021 --- -# Distributed Tracing +# .NET Distributed Tracing Concepts -Distributed tracing is a diagnostic technique that helps engineers localize failures and +Distributed tracing is a diagnostic technique that helps engineers localize failures and performance issues within applications, especially those that may be distributed across multiple machines or processes. See the [Distributed Tracing Overview](distributed-tracing.md) for general information about where distributed tracing is useful and example code to get started. -## Concepts - ### Traces and Activities -Each time a new request is received by an application a new trace can be initiated. In +Each time a new request is received by an application it can be associated with a trace. In application components written in .NET, units of work in a trace are represented by instances of and the trace as a whole forms a tree of these Activities, potentially spanning across many distinct processes. The first @@ -24,15 +22,13 @@ Activity created for a new request forms the root of the trace tree and it track duration and success/failure handling the request. Child activities can be optionally created to sub-divide the work into different steps that can be tracked individually. For example given an Activity that tracked a specific inbound HTTP request in a web server, -child activites could be created to track each of the database queries that was necessary to -complete the request. This allows the duration and success for each query to be recorded independently. -Activities can record other information for each unit of work such as a name ( -), name-value pairs -called tags ( -), and events ( -). The name identifies -the type of work being performed, tags can record descriptive parameters of the work, and events -are a simple logging mechanism to record timestamped diagnostic messages. +child activites could be created to track each of the database queries that were necessary to +complete the request. This allows the duration and success for each query to be recorded independently. +Activities can record other information for each unit of work such as +, name-value pairs +called , and . The +name identifies the type of work being performed, tags can record descriptive parameters of the work, +and events are a simple logging mechanism to record timestamped diagnostic messages. > [!NOTE] > Another common industry name for units of work in a distributed trace are 'Spans'. @@ -45,42 +41,46 @@ Parent-Child relationships between Activities in the distributed trace tree are using unique IDs. .NET's implementation of distributed tracing supports two ID schemes, the W3C standard [TraceContext](https://www.w3.org/TR/trace-context/) which is the default in .NET 5 and an older .NET convention called 'Hierarchical' that is available for backwards compatibility. - controls which + controls which ID scheme is used. In the W3C TraceContext standard every trace is assigned a globally unique 16 -byte trace-id () and -every Activity within the trace is assigned a unique 8 byte span-id ( -). Each Activity -records the trace-id, its own span-id, and the span-id of its parent ( -). Because +byte trace-id () and +every Activity within the trace is assigned a unique 8 byte span-id +(). Each Activity +records the trace-id, its own span-id, and the span-id of its parent +(). Because distributed traces can track work across process boundaries parent and child Activities may not be in the same process. The combination of a trace-id and parent span-id can uniquely -identify the parent Activity globally, regardless of what process it resides in. The - property will also -reference the parent Activity whenever the parent resides in the same process. +identify the parent Activity globally, regardless of what process it resides in. + + controls which +ID format is used for starting new traces, but by default adding a new Activity to an existing +trace uses whatever format the parent Activity is using. +Setting +to true overrides this behavior and creates all new Activities with the DefaultIdFormat, even +when the parent uses a different ID format. ### Starting and stopping Activities Each thread in a process may have a corresponding Activity object that is tracking the work -occuring on that thread, accessible via -. The current activity +occuring on that thread, accessible via +. The current activity automatically flows along all synchronous calls on a thread as well as following async calls that are processed on different threads. If Activity A is the current activity on a thread and code starts a new Activity B then B becomes the new current activity on that thread. By default activity B will also treat Activity A as its parent. When Activity B is later stopped activity A will be restored as the current Activity on the thread. When an Activity is started it -captures the current time as the -. When it -stops is calculated +captures the current time as the +. When it +stops is calculated as the difference between the current time and the start time. ### Coordinating across process boundaries In order to track work across process boundaries Activity parent IDs need to be transmitted across the network so that the receiving process can create Activities that refer to them. When using -W3C standard Activity IDs () -.NET will also use [W3C standard networking protocols](https://www.w3.org/TR/trace-context/) to -transmit this information. When using the - ID format +W3C TraceContext ID format .NET will also use the HTTP headers recommended by +[the standard](https://www.w3.org/TR/trace-context/) to transmit this information. When using the + ID format .NET uses a custom request-id HTTP header to transmit the ID. Unlike many other language runtimes .NET in-box libraries natively understand how to decode and encode Activity IDs on HTTP messages as well as how to flow the ID through sychronous and asynchronous calls. This means that .NET @@ -89,49 +89,49 @@ automatically, with no special coding by the app developer nor 3rd party library 3rd party libraries may add support for transmitting IDs over non-HTTP message protocols or supporting custom encoding conventions for HTTP. -### Logging Activities +### Collecting traces -After creating the objects the -app developer typically wants to log the tracing information to a persistant store so it can be -later reviewed by engineers on-demand. There are several telemetry collection libraries that can -do this such as -[Application Insights](https://docs.microsoft.com/en-us/azure/azure-monitor/app/distributed-tracing), +Instrumented code can create objects +as part of a distributed trace, but the information in these objects needs to be transmitted +and serialized in a centralized persistant store so that the entire trace can be usefully reviewed +later. There are several telemetry collection libraries that can do this task such as +[Application Insights](https://docs.microsoft.com/azure/azure-monitor/app/distributed-tracing), [OpenTelemetry](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/trace/getting-started/README.md), -or a library provided by a 3rd party logging or APM vendor. Alternately developers can author -their own custom Activity logging by using - or +or a library provided by a 3rd party telemetry or APM vendor. Alternately developers can author +their own custom Activity telemetry collection by using + or . ActivityListener -supports logging any Activity regardless whether the logging library has any a-priori knowledge -which makes it a simple and flexible general purpose solution. By contrast using DiagnosticListener -is a more complex scenario that requires the Activity creator to opt-in by invoking a specific API, -, and -the logging library needs to know the exact naming information that the Activity creator chose to -use when starting it. Using DiagnosticSource and DiagnosticListener allows the Activity creator -and Activity logger to exchange arbitrary .NET objects and establish customized information passing +supports observing any Activity regardless whether the developer has any a-priori knowledge about it. +This makes ActivityListener a simple and flexible general purpose solution. By contrast using +DiagnosticListener is a more complex scenario that requires the instrumented code to opt-in by +invoking and +the collection library needs to know the exact naming information that the instrumented code +used when starting it. Using DiagnosticSource and DiagnosticListener allows the creator +and listener to exchange arbitrary .NET objects and establish customized information passing conventions. ### Sampling For improved performance in high throughput applications, distributed tracing on .NET supports -sampling only a subset of requests rather than logging all of them. For activites created with -the recommended newer +sampling only a subset of requests rather than recording all of them. For activites created with +the recommended API, logging libraries can control sampling with the callback. The logging library can elect not to create the Activity at all, to create it with minimal information necessary to propagate distributing tracing IDs, or to populate it with complete diagnostic information. These choices trade-off increasing performance overhead for increasing diagnostic utility. Activities that are started using the older pattern of invoking - and + and may -also support DiagnosticListener sampling by first calling +also support DiagnosticListener sampling by first calling . Even when capturing full diagnostic information the .NET -implementation is designed to be fast - coupled with an efficient logger an Activity can be -created, populated, and logged in about a microsecond on modern hardware. Sampling +implementation is designed to be fast - coupled with an efficient collector an Activity can be +created, populated, and transmitted in about a microsecond on modern hardware. Sampling can reduce the instrumentation cost to less than 100 nanoseconds for each Activity that isn't -logged. +recorded. ## Next steps See the [Distributed Tracing Overview](distributed-tracing.md) for example code to get started -using distributed tracing in .NET applications. \ No newline at end of file +using distributed tracing in .NET applications. diff --git a/docs/core/diagnostics/distributed-tracing-instrumentation-walkthroughs.md b/docs/core/diagnostics/distributed-tracing-instrumentation-walkthroughs.md index 57a8c800c58d7..de585a27350f1 100644 --- a/docs/core/diagnostics/distributed-tracing-instrumentation-walkthroughs.md +++ b/docs/core/diagnostics/distributed-tracing-instrumentation-walkthroughs.md @@ -5,10 +5,9 @@ ms.topic: tutorial ms.date: 03/14/2021 --- -# Adding distrubuted tracing instrumentation +# Adding distributed tracing instrumentation -**This article applies to: ✔️** .NET Core 5.0 and later versions **or** any .NET application using the -[DiagnosticSource NuGet package](https://www.nuget.org/packages/System.Diagnostics.DiagnosticSource/5.0.1) version 5 or later +**This article applies to: ✔️** .NET Core 2.1 and later versions **and** .NET Framework 4.5 and later versions .NET applications can be instrumented using the API to produce distributed tracing telemetry. Some instrumentation is built-in to standard .NET libraries but you may want to add more to make @@ -22,7 +21,7 @@ produced by this instrumentation. ## An initial app -First you will create a sample app that records telemetry using OpenTelemetry, but doesn't yet have any instrumentation. +First you will create a sample app that collects telemetry using OpenTelemetry, but doesn't yet have any instrumentation. ```dotnetcli dotnet new console @@ -30,12 +29,13 @@ dotnet new console Applications that target .NET 5 and later already have the necessary distributed tracing APIs included. For apps targeting older .NET versions add the [System.Diagnostics.DiagnosticSource NuGet package](https://www.nuget.org/packages/System.Diagnostics.DiagnosticSource/) -version 5.0.1 or greater. +version 5 or greater. + ```dotnetcli -dotnet add package System.Diagnostics.DiagnosticSource --version 5.0.1 +dotnet add package System.Diagnostics.DiagnosticSource ``` -Add the [OpenTelemetry](https://www.nuget.org/packages/OpenTelemetry/) and +Add the [OpenTelemetry](https://www.nuget.org/packages/OpenTelemetry/) and [OpenTelemetry.Exporter.Console](https://www.nuget.org/packages/OpenTelemetry.Exporter.Console/) NuGet packages which will be used to collect the telemetry. @@ -58,10 +58,7 @@ namespace Sample.DistributedTracing static async Task Main(string[] args) { using var tracerProvider = Sdk.CreateTracerProviderBuilder() - .SetSampler(new AlwaysOnSampler()) - // Add more libraries .AddSource("Sample.DistributedTracing") - // Add more exporters .AddConsoleExporter() .Build(); @@ -89,7 +86,8 @@ namespace Sample.DistributedTracing } ``` -Running the app does not log anything yet +The app has no instrumentation yet so there is no trace information to display: + ```dotnetcli > dotnet run Example work done @@ -97,7 +95,7 @@ Example work done #### Best Practices -Only app developers need to reference an optional 3rd party library for exporting the +Only app developers need to reference an optional 3rd party library for collecting the distributed trace telemetry, such as OpenTelemetry in this example. .NET library authors can exclusively rely on APIs in System.Diagnostics.DiagnosticSource which is part of .NET runtime. This ensures that libraries will run in a wide range of .NET apps, regardless @@ -114,7 +112,7 @@ Applications and libraries add distributed tracing instrumentation using the First create an instance of ActivitySource. ActivitySource provides APIs to create and start Activity objects. Add the static ActivitySource variable above Main() and -`using System.Diagnostics;` to the using statements. +`using System.Diagnostics;` to the using statements. ```csharp using OpenTelemetry; @@ -144,7 +142,7 @@ enable and disable the Activity telemetry in the sources independently. - The source name passed to the constructor has to be unique to avoid the conflicts with any other sources. It is recommended to use a hierarchical name that contains the assembly name and optionally a component name if there are multiple sources within the same assembly. For example, `Microsoft.AspNetCore.Hosting`. If an assembly -is adding instrumentation for code in a 2nd independent assembly, the name should be based on the +is adding instrumentation for code in a 2nd independent assembly, the name should be based on the assembly that defines the ActivitySource, not the assembly whose code is being instrumented. - The version parameter is optional. It is recommended to provide the version in case you release multiple @@ -189,15 +187,15 @@ Example work done #### Notes -- creates and starts +- creates and starts the activity at the same time. The listed code pattern is using the `using` block which automatically disposes the created Activity object after executing the block. Disposing the Activity object will stop it so the code doesn't need to explicitly call . That simplifies the coding pattern. -- internally determines if +- internally determines if there are any listeners recording the Activity. If there are no registered listeners or there are listeners which -are not interested, `ActivitySource.StartActivity` will return `null` and avoid creating the Activity object. This +are not interested, StartActivity() will return `null` and avoid creating the Activity object. This is a performance optimization so that the code pattern can still be used in functions that are called very frequently. @@ -237,10 +235,11 @@ Example work done #### Best Practices -- As mentioned above, `activity` returned by -may be null. The null-coallescing operator ?. in C# is a very convenient short-hand to only invoke - if activity is not null. The behavior is identical to +- As mentioned above, `activity` returned by +may be null. The null-coallescing operator ?. in C# is a convenient short-hand to only invoke + if activity is not null. The behavior is identical to writing: + ```csharp if(activity != null) { @@ -252,9 +251,9 @@ if(activity != null) [conventions](https://github.com/open-telemetry/opentelemetry-specification/tree/main/specification/trace/semantic_conventions) for setting Tags on Activities that represent common types of application work. -- If you are instrumenting functions with high performance requirements, +- If you are instrumenting functions with high performance requirements, is a hint that indicates whether any -of the code logging Activities intends to read auxilliary information such as Tags. If no logger will read it then there +of the code listening to Activities intends to read auxilliary information such as Tags. If no listener will read it then there is no need for the instrumented code to spend CPU cycles populating it. For simplicity this sample doesn't apply that optimization. @@ -301,10 +300,10 @@ Example work done - Events are stored in an in-memory list until they can be transmitted which makes this mechanism only suitable for recording a modest number of events. For a large or unbounded volume of events using a logging API focused on this task -such as [ILogger](../../aspnet/core/fundamentals/logging) is a better choice. ILogger also ensures that the logging -information will be available regardless whether the app developer opts to use distributed tracing. ILogger supports -automatically capturing the active Activity IDs so messages logged via that API can still be correlated with the -distributed trace. +such as [ILogger](https://docs.microsoft.com/aspnet/core/fundamentals/logging/) is a better choice. ILogger also ensures +that the logging information will be available regardless whether the app developer opts to use distributed tracing. +ILogger supports automatically capturing the active Activity IDs so messages logged via that API can still be correlated +with the distributed trace. ## Optional: Adding Status @@ -342,7 +341,9 @@ Update DoSomeWork() to set status: Activities can be nested to describe portions of a larger unit of work. This can be particularly valuable around portions of code that might not execute quickly or to better localize failures that come from specific external -dependencies. +dependencies. Although this sample uses an Activity in every method, that is solely because extra code has been +minimized. In a larger and more realistic project using an Activity in every method would produce extremely +verbose traces so it is not recommended. Update StepOne and StepTwo to add more tracing around these separate steps: @@ -404,9 +405,44 @@ Example work done ``` Notice that both StepOne and StepTwo include a ParentId that refers to SomeWork. The console is -not a great visualization of nested trees of work, but many GUI viewers such as +not a great visualization of nested trees of work, but many GUI viewers such as [Zipkin](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/src/OpenTelemetry.Exporter.Zipkin/README.md) can show this as a Gantt chart: [![Zipkin Gantt chart](media/zipkin-nested-activities.jpg)](media/zipkin-nested-activities.jpg) +### Optional: ActivityKind + +Activities have an property which +describes the relationship between the Activity, its parent and its children. By default all new Activities are +set to which is appropriate for Activities that are an +internal operation within an application with no remote parent or children. Other kinds can be set using the +kind parameter on +. See + for other options. + +### Optional: Links + +When work occurs in batch processing systems a single Activity might represent work on behalf of many +different requests simultaneously, each of which has its own trace-id. Although Activity is restricted +to have a single parent, it can link to additional trace-ids using +. Each ActivityLink is +populated with an that +stores ID information about the Activity being linked to. ActivityContext can be retrieved from in-process +Activity objects using or +it can be parsed from serialized id information using +. + +```csharp +void DoBatchWork(ActivityContext[] requestContexts) +{ + // Assume each context in requestContexts encodes the trace-id that was sent with a request + using(Activity activity = s_source.StartActivity(name: "BigBatchOfWork", + kind: ActivityKind.Internal, + parentContext: null, + links: requestIds.Select(ctx => new ActivityLink(ctx)) + { + // do the batch of work here + } +} +``` diff --git a/docs/core/diagnostics/distributed-tracing.md b/docs/core/diagnostics/distributed-tracing.md index 9abe3a1cfe5b0..c7da10865fe90 100644 --- a/docs/core/diagnostics/distributed-tracing.md +++ b/docs/core/diagnostics/distributed-tracing.md @@ -5,7 +5,7 @@ ms.date: 03/15/2021 --- # .NET Distributed Tracing -Distributed tracing is a diagnostic technique that helps engineers localize failures and +Distributed tracing is a diagnostic technique that helps engineers localize failures and performance issues within applications, especially those that may be distributed across multiple machines or processes. This technique tracks requests through an application correlating together work done by different application components and separating it from @@ -17,22 +17,22 @@ logging messages produced by each step as it ran. ## Getting started for .NET app developers -Many .NET libraries are instrumented to produce distributed tracing information out of the box -but this information needs to be captured into a log before it will be useful. Typically app -developers will select a telemetry service that stores this logging information for them and +Key .NET libraries are instrumented to produce distributed tracing information automatically +but this information needs to be collected and stored so that it will be available for review later. +Typically app developers will select a telemetry service that stores this trace information for them and then use a corresponding library to transmit the distributed tracing telemetry to their chosen service. [OpenTelemetry](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/trace/getting-started/README.md) -is a vendor neutral library that supports several services, -[Application Insights](https://docs.microsoft.com/en-us/azure/azure-monitor/app/distributed-tracing) +is a vendor neutral library that supports several services, +[Application Insights](https://docs.microsoft.com/azure/azure-monitor/app/distributed-tracing) is provided by Microsoft, and there are many high quality 3rd party APM vendors that offer integrated .NET solutions. - - [Understand distributed tracing concepts](distributed-tracing-concepts.md) - - Guides - - [Collecting Distributed Traces with Application Insights](distributed-tracing-collection-walkthroughs.md#collect-using-application-insights) - - [Collecting Distributed Traces with OpenTelemetry](distributed-tracing-collection-walkthroughs.md#collect-using-opentelemetry) - - [Collecting a custom distributed trace log](distributed-tracing-collection-walkthroughs.md#collect-using-a-custom-logging-implementation) - - [Adding custom distributed trace instrumentation](distributed-tracing-instrumentation-walkthroughs.md) +- [Understand distributed tracing concepts](distributed-tracing-concepts.md) +- Guides + - [Collect distributed traces with Application Insights](distributed-tracing-collection-walkthroughs.md#collect-traces-using-application-insights) + - [Collect distributed traces with OpenTelemetry](distributed-tracing-collection-walkthroughs.md#collect-traces-using-opentelemetry) + - [Collect distributed traces with custom logic](distributed-tracing-collection-walkthroughs.md#collect-traces-using-custom-logic) + - [Adding custom distributed trace instrumentation](distributed-tracing-instrumentation-walkthroughs.md) For 3rd party telemetry collection services follow the setup instructions provided by the vendor. @@ -43,8 +43,6 @@ with how it is produced. If you believe .NET app developers that use your librar appreciate seeing the work that it does detailed in a distributed trace then you should add distributed tracing instrumentation to support it. - - [Understand distributed tracing concepts](distributed-tracing-concepts.md) - - Guides - - [Adding custom distributed trace instrumentation](distributed-tracing-instrumentation-walkthroughs.md) - - +- [Understand distributed tracing concepts](distributed-tracing-concepts.md) +- Guides + - [Adding custom distributed trace instrumentation](distributed-tracing-instrumentation-walkthroughs.md)