Skip to content

Commit

Permalink
Add sampling by trace resource
Browse files Browse the repository at this point in the history
  • Loading branch information
marcotc committed Apr 11, 2024
1 parent 07c75b8 commit 7398792
Show file tree
Hide file tree
Showing 7 changed files with 142 additions and 37 deletions.
2 changes: 1 addition & 1 deletion docs/GettingStarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -2132,7 +2132,7 @@ For example, if `tracing.sampling.default_rate` is configured by [Remote Configu
| `tracing.sampler` | | `nil` | Advanced usage only. Sets a custom `Datadog::Tracing::Sampling::Sampler` instance. If provided, the tracer will use this sampler to determine sampling behavior. See [Application-side sampling](#application-side-sampling) for details. |
| `tracing.sampling.default_rate` | `DD_TRACE_SAMPLE_RATE` | `nil` | Sets the trace sampling rate between `0.0` (0%) and `1.0` (100%). See [Application-side sampling](#application-side-sampling) for details. |
| `tracing.sampling.rate_limit` | `DD_TRACE_RATE_LIMIT` | `100` (per second) | Sets a maximum number of traces per second to sample. Set a rate limit to avoid the ingestion volume overages in the case of traffic spikes. |
| `tracing.sampling.rules` | `DD_TRACE_SAMPLING_RULES` | `nil` | Sets trace-level sampling rules, matching against the local root span. The format is a `String` with JSON, containing an Array of Objects. Each Object must have a float attribute `sample_rate` (between 0.0 and 1.0, inclusive), and optionally `name` and `service` string attributes. `name` and `service` control to which traces this sampling rule applies; if both are absent, then this rule applies to all traces. Rules are evaluted in order of declartion in the array; only the first to match is applied. If none apply, then `tracing.sampling.default_rate` is applied. |
| `tracing.sampling.rules` | `DD_TRACE_SAMPLING_RULES` | `nil` | Sets trace-level sampling rules, matching against the local root span. The format is a `String` with JSON, containing an Array of Objects. Each Object must have a float attribute `sample_rate` (between 0.0 and 1.0, inclusive), and optionally `name`, `service`, and `resource` string attributes. `name`, `service`, and `resource` control to which traces this sampling rule applies; if they are all absent, then this rule applies to all traces. Rules are evaluted in order of declartion in the array; only the first to match is applied. If none apply, then `tracing.sampling.default_rate` is applied. |
| `tracing.sampling.span_rules` | `DD_SPAN_SAMPLING_RULES`,`ENV_SPAN_SAMPLING_RULES_FILE` | `nil` | Sets [Single Span Sampling](#single-span-sampling) rules. These rules allow you to keep spans even when their respective traces are dropped. |
| `tracing.trace_id_128_bit_generation_enabled` | `DD_TRACE_128_BIT_TRACEID_GENERATION_ENABLED` | `true` | `true` to generate 128 bits trace ID and `false` to generate 64 bits trace ID |
| `tracing.report_hostname` | `DD_TRACE_REPORT_HOSTNAME` | `false` | Adds hostname tag to traces. |
Expand Down
7 changes: 4 additions & 3 deletions lib/datadog/tracing/sampling/matcher.rb
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,21 @@ def ===(other)
end
end.new

attr_reader :name, :service
attr_reader :name, :service, :resource

# @param name [String,Regexp,Proc] Matcher for case equality (===) with the trace name,
# defaults to always match
# @param service [String,Regexp,Proc] Matcher for case equality (===) with the service name,
# defaults to always match
def initialize(name: MATCH_ALL, service: MATCH_ALL)
def initialize(name: MATCH_ALL, service: MATCH_ALL, resource: MATCH_ALL)
super()
@name = name
@service = service
@resource = resource
end

def match?(trace)
name === trace.name && service === trace.service
name === trace.name && service === trace.service && resource === trace.resource
end
end

Expand Down
7 changes: 5 additions & 2 deletions lib/datadog/tracing/sampling/rule.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@ class SimpleRule < Rule
# @param service [String,Regexp,Proc] Matcher for case equality (===) with the service name,
# defaults to always match
# @param sample_rate [Float] Sampling rate between +[0,1]+
def initialize(name: SimpleMatcher::MATCH_ALL, service: SimpleMatcher::MATCH_ALL, sample_rate: 1.0)
def initialize(
name: SimpleMatcher::MATCH_ALL, service: SimpleMatcher::MATCH_ALL,
resource: SimpleMatcher::MATCH_ALL, sample_rate: 1.0
)
# We want to allow 0.0 to drop all traces, but {Datadog::Tracing::Sampling::RateSampler}
# considers 0.0 an invalid rate and falls back to 100% sampling.
#
Expand All @@ -64,7 +67,7 @@ def initialize(name: SimpleMatcher::MATCH_ALL, service: SimpleMatcher::MATCH_ALL
sampler = RateSampler.new
sampler.sample_rate = sample_rate

super(SimpleMatcher.new(name: name, service: service), sampler)
super(SimpleMatcher.new(name: name, service: service, resource: resource), sampler)
end
end
end
Expand Down
1 change: 1 addition & 0 deletions lib/datadog/tracing/sampling/rule_sampler.rb
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ def self.parse(rules, rate_limit, default_sample_rate)
kwargs = {
name: rule['name'],
service: rule['service'],
resource: rule['resource'],
sample_rate: sample_rate,
}

Expand Down
106 changes: 85 additions & 21 deletions spec/datadog/tracing/sampling/matcher_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,15 @@
require 'datadog/tracing/sampling/matcher'

RSpec.describe Datadog::Tracing::Sampling::SimpleMatcher do
let(:span_op) { Datadog::Tracing::SpanOperation.new(span_name, service: span_service) }
let(:span_name) { 'operation.name' }
let(:span_service) { nil }
let(:trace_op) do
Datadog::Tracing::TraceOperation.new(name: trace_name, service: trace_service, resource: trace_resource)
end
let(:trace_name) { 'operation.name' }
let(:trace_service) { 'test-service' }
let(:trace_resource) { 'test-resource' }

describe '#match?' do
subject(:match?) { rule.match?(span_op) }
subject(:match?) { rule.match?(trace_op) }

context 'with a name matcher' do
let(:rule) { described_class.new(name: name) }
Expand All @@ -29,7 +32,7 @@

context 'with a string' do
context 'matching' do
let(:name) { span_name.to_s }
let(:name) { trace_name.to_s }

it { is_expected.to eq(true) }
end
Expand All @@ -43,7 +46,7 @@

context 'with a proc' do
context 'matching' do
let(:name) { ->(n) { n == span_name } }
let(:name) { ->(n) { n == trace_name } }

it { is_expected.to eq(true) }
end
Expand All @@ -59,8 +62,8 @@
context 'with a service matcher' do
let(:rule) { described_class.new(service: service) }

context 'when span service name is present' do
let(:span_service) { 'service-1' }
context 'when trace service name is present' do
let(:trace_service) { 'service-1' }

context 'with a regexp' do
context 'matching' do
Expand All @@ -78,7 +81,7 @@

context 'with a string' do
context 'matching' do
let(:service) { span_service.to_s }
let(:service) { trace_service.to_s }

it { is_expected.to eq(true) }
end
Expand All @@ -92,7 +95,7 @@

context 'with a proc' do
context 'matching' do
let(:service) { ->(n) { n == span_service } }
let(:service) { ->(n) { n == trace_service } }

it { is_expected.to eq(true) }
end
Expand All @@ -105,44 +108,105 @@
end
end

context 'when span service is not present' do
context 'when trace service is not present' do
let(:trace_service) { nil }
let(:service) { /.*/ }

it { is_expected.to eq(false) }
end
end

context 'with name and service matchers' do
let(:rule) { described_class.new(name: name, service: service) }
context 'with a resource matcher' do
let(:rule) { described_class.new(resource: resource) }

context 'when trace resource is present' do
let(:trace_resource) { 'resource-1' }

context 'with a regexp' do
context 'matching' do
let(:resource) { /resource-.*/ }

it { is_expected.to eq(true) }
end

context 'not matching' do
let(:resource) { /name-.*/ }

it { is_expected.to eq(false) }
end
end

context 'with a string' do
context 'matching' do
let(:resource) { 'resource-1' }

it { is_expected.to eq(true) }
end

context 'not matching' do
let(:resource) { 'not-resource' }

it { is_expected.to eq(false) }
end
end

context 'with a proc' do
context 'matching' do
let(:resource) { ->(n) { n == 'resource-1' } }

it { is_expected.to eq(true) }
end

context 'not matching' do
let(:resource) { ->(_n) { false } }

it { is_expected.to eq(false) }
end
end
end

context 'when trace resource is not present' do
let(:trace_resource) { nil }
let(:resource) { /.*/ }

it { is_expected.to eq(false) }
end
end

context 'with name, service, resource matchers' do
let(:rule) { described_class.new(name: name, service: service, resource: resource) }

let(:name) { /.*/ }
let(:service) { /.*/ }
let(:resource) { /.*/ }

context 'when span service name is present' do
let(:span_service) { 'service-1' }
context 'when trace service name is present' do
let(:trace_service) { 'service-1' }

it { is_expected.to eq(true) }
end

context 'when span service is not present' do
context 'when trace service is not present' do
let(:trace_service) { nil }

it { is_expected.to eq(false) }
end
end
end
end

RSpec.describe Datadog::Tracing::Sampling::ProcMatcher do
let(:span_op) { Datadog::Tracing::SpanOperation.new(span_name, service: span_service) }
let(:span_name) { 'operation.name' }
let(:span_service) { nil }
let(:trace_op) { Datadog::Tracing::SpanOperation.new(trace_name, service: trace_service) }
let(:trace_name) { 'operation.name' }
let(:trace_service) { nil }

describe '#match?' do
subject(:match?) { rule.match?(span_op) }
subject(:match?) { rule.match?(trace_op) }

let(:rule) { described_class.new(&block) }

context 'with matching block' do
let(:block) { ->(name, service) { name == span_name && service == span_service } }
let(:block) { ->(name, service) { name == trace_name && service == trace_service } }

it { is_expected.to eq(true) }
end
Expand Down
8 changes: 8 additions & 0 deletions spec/datadog/tracing/sampling/rule_sampler_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,14 @@
end
end

context 'and resource' do
let(:rule) { { sample_rate: 0.1, resource: 'test-resource' } }

it 'parses resource matcher' do
expect(actual_rule.matcher.resource).to eq('test-resource')
end
end

context 'with multiple rules' do
let(:rules) { [{ sample_rate: 0.1 }, { sample_rate: 0.2 }] }

Expand Down
48 changes: 38 additions & 10 deletions spec/datadog/tracing/sampling/rule_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,27 @@
require 'datadog/core'
require 'datadog/tracing/sampling/matcher'
require 'datadog/tracing/sampling/rule'
require 'datadog/tracing/span_operation'
require 'datadog/tracing/trace_operation'

RSpec.describe Datadog::Tracing::Sampling::Rule do
let(:span_op) { Datadog::Tracing::SpanOperation.new(span_name, service: span_service) }
let(:span_name) { 'operation.name' }
let(:span_service) { nil }
let(:trace_op) do
Datadog::Tracing::TraceOperation.new(name: trace_name, service: trace_service, resource: trace_resource)
end
let(:trace_name) { 'operation.name' }
let(:trace_service) { 'test-service' }
let(:trace_resource) { 'test-resource' }

let(:rule) { described_class.new(matcher, sampler) }
let(:matcher) { instance_double(Datadog::Tracing::Sampling::Matcher) }
let(:sampler) { instance_double(Datadog::Tracing::Sampling::Sampler) }

describe '#match?' do
subject(:match) { rule.match?(span_op) }
subject(:match) { rule.match?(trace_op) }

let(:matched) { true }

before do
allow(matcher).to receive(:match?).with(span_op).and_return(matched)
allow(matcher).to receive(:match?).with(trace_op).and_return(matched)
end

context 'with matching span operation' do
Expand Down Expand Up @@ -50,26 +53,51 @@
end

describe '#sample?' do
subject(:sample?) { rule.sample?(span_op) }
subject(:sample?) { rule.sample?(trace_op) }

let(:sample) { double }

before do
allow(sampler).to receive(:sample?).with(span_op).and_return(sample)
allow(sampler).to receive(:sample?).with(trace_op).and_return(sample)
end

it { is_expected.to be(sample) }
end

describe '#sample_rate' do
subject(:sample_rate) { rule.sample_rate(span_op) }
subject(:sample_rate) { rule.sample_rate(trace_op) }

let(:sample_rate_value) { double }

before do
allow(sampler).to receive(:sample_rate).with(span_op).and_return(sample_rate_value)
allow(sampler).to receive(:sample_rate).with(trace_op).and_return(sample_rate_value)
end

it { is_expected.to be(sample_rate_value) }
end
end

RSpec.describe Datadog::Tracing::Sampling::SimpleRule do
let(:trace_op) do
Datadog::Tracing::TraceOperation.new(name: trace_name, service: trace_service, resource: trace_resource)
end
let(:trace_name) { 'operation.name' }
let(:trace_service) { 'test-service' }
let(:trace_resource) { 'test-resource' }

describe '#initialize' do
subject(:rule) { described_class.new(name: name, service: service, resource: resource, sample_rate: sample_rate) }

let(:name) { double('name') }
let(:service) { double('service') }
let(:resource) { double('resource') }
let(:sample_rate) { 0.123 }

it 'initializes with the correct values' do
expect(rule.matcher.name).to eq(name)
expect(rule.matcher.service).to eq(service)
expect(rule.matcher.resource).to eq(resource)
expect(rule.sampler.sample_rate).to eq(sample_rate)
end
end
end

0 comments on commit 7398792

Please sign in to comment.