Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Health Metrics for data-pipeline #618

Closed
wants to merge 25 commits into from

Conversation

ajgajg1134
Copy link
Contributor

What does this PR do?

A brief description of the change being made with this pull request.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@ajgajg1134 ajgajg1134 changed the title move dogstatsd-client to separate crate so it can be shared Health Metrics for data-pipeline Sep 10, 2024
Cargo.toml Outdated Show resolved Hide resolved
dogstatsd-client/Cargo.toml Outdated Show resolved Hide resolved
@@ -1,4 +1,4 @@
// Copyright 2021-Present Datadog, Inc. https://www.datadoghq.com/
// Copyright 2024-Present Datadog, Inc. https://www.datadoghq.com/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no clue what the correct answer is, but should we be modifying the license headers like this? I'm not sure what is legally correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iirc it doesn't strictly matter as long as the header is there when the file is created

@@ -213,23 +231,36 @@ impl TraceExporter {
return Ok(String::from("{}"));
}

// todo: do we need to modify the client to allow for &str to avoid allocating a String?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client should probably have some sort of ActionType enum implemented instead of passing strings like this. Partially for the allocation cost, and partially to reduce the likelihood of typos.

@codecov-commenter
Copy link

codecov-commenter commented Sep 12, 2024

Codecov Report

Attention: Patch coverage is 90.34483% with 14 lines in your changes missing coverage. Please review.

Project coverage is 73.28%. Comparing base (89f48d4) to head (5532b66).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #618      +/-   ##
==========================================
+ Coverage   73.05%   73.28%   +0.23%     
==========================================
  Files         252      252              
  Lines       36093    36153      +60     
==========================================
+ Hits        26366    26496     +130     
+ Misses       9727     9657      -70     
Components Coverage Δ
crashtracker 20.48% <ø> (-0.06%) ⬇️
datadog-alloc 98.73% <ø> (ø)
data-pipeline 94.05% <92.68%> (+3.93%) ⬆️
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 83.08% <ø> (ø)
ddcommon-ffi 69.52% <ø> (ø)
ddtelemetry 59.10% <ø> (ø)
ipc 83.63% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 38.46% <0.00%> (-1.67%) ⬇️
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.36% <ø> (ø)
tinybytes 91.66% <ø> (ø)
trace-mini-agent 72.49% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.30% <ø> (+0.41%) ⬆️

@pr-commenter
Copy link

pr-commenter bot commented Sep 17, 2024

Benchmarks

Comparison

Benchmark execution time: 2024-09-17 18:39:54

Comparing candidate commit 5532b66 in PR branch andrew.glaude/dogstatsd-client with baseline commit 89f48d4 in branch main.

Found 7 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/37828224631000521389798

  • 🟩 execution_time [-7.844µs; -7.474µs] or [-7.591%; -7.234%]
  • 🟩 throughput [+756545.657op/s; +792908.700op/s] or [+7.817%; +8.192%]

scenario:credit_card/is_card_number_no_luhn/ 3782-8224-6310-005

  • 🟩 execution_time [-8.635µs; -8.166µs] or [-8.279%; -7.830%]
  • 🟩 throughput [+815663.845op/s; +860852.031op/s] or [+8.505%; +8.976%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

  • 🟩 execution_time [-7.714µs; -7.403µs] or [-7.476%; -7.175%]
  • 🟩 throughput [+750763.053op/s; +781456.313op/s] or [+7.746%; +8.063%]

scenario:normalization/normalize_trace/test_trace

  • 🟩 execution_time [-16.547ns; -11.508ns] or [-5.209%; -3.623%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 16.281µs 24.064µs ± 11.158µs 16.512µs ± 0.136µs 32.858µs 40.847µs 44.101µs 91.949µs 456.85% 2.434 11.040 46.25% 0.789µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [22.517µs; 25.610µs] or [-6.426%; +6.426%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 506.944µs 507.533µs ± 0.292µs 507.493µs ± 0.186µs 507.718µs 507.950µs 508.368µs 509.099µs 0.32% 1.369 5.178 0.06% 0.021µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1964254.270op/s 1970315.192op/s ± 1131.818op/s 1970468.587op/s ± 721.513op/s 1971107.335op/s 1971907.006op/s 1972407.850op/s 1972604.927op/s 0.11% -1.360 5.127 0.06% 80.032op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 468.454µs 469.246µs ± 0.347µs 469.225µs ± 0.238µs 469.460µs 469.847µs 470.195µs 470.284µs 0.23% 0.451 0.145 0.07% 0.025µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2126376.605op/s 2131077.323op/s ± 1573.899op/s 2131172.071op/s ± 1082.333op/s 2132266.931op/s 2133367.695op/s 2133859.468op/s 2134680.114op/s 0.16% -0.447 0.139 0.07% 111.291op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 180.121µs 180.736µs ± 0.325µs 180.674µs ± 0.229µs 180.935µs 181.387µs 181.557µs 181.592µs 0.51% 0.594 -0.319 0.18% 0.023µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5506856.439op/s 5532962.706op/s ± 9946.815op/s 5534832.932op/s ± 7015.701op/s 5540704.703op/s 5546249.097op/s 5548551.698op/s 5551832.518op/s 0.31% -0.587 -0.330 0.18% 703.346op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 43.511µs 43.706µs ± 0.074µs 43.709µs ± 0.050µs 43.756µs 43.821µs 43.862µs 43.918µs 0.48% -0.013 -0.181 0.17% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 22769728.935op/s 22880258.435op/s ± 38953.769op/s 22878530.283op/s ± 26241.686op/s 22907102.783op/s 22944443.785op/s 22970249.609op/s 22982468.716op/s 0.45% 0.023 -0.183 0.17% 2754.447op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.918µs 50.235µs ± 0.149µs 50.287µs ± 0.071µs 50.340µs 50.408µs 50.474µs 50.520µs 0.46% -0.687 -0.729 0.30% 0.011µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19794204.717op/s 19906803.725op/s ± 59216.627op/s 19885875.883op/s ± 27977.618op/s 19954034.173op/s 20016178.430op/s 20021720.634op/s 20032671.890op/s 0.74% 0.694 -0.725 0.30% 4187.248op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [507.493µs; 507.574µs] or [-0.008%; +0.008%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1970158.333op/s; 1970472.051op/s] or [-0.008%; +0.008%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [469.198µs; 469.295µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2130859.195op/s; 2131295.450op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [180.691µs; 180.781µs] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5531584.173op/s; 5534341.239op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [43.696µs; 43.716µs] or [-0.024%; +0.024%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22874859.817op/s; 22885657.053op/s] or [-0.024%; +0.024%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [50.214µs; 50.255µs] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [19898596.870op/s; 19915010.580op/s] or [-0.041%; +0.041%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 140.757µs 141.833µs ± 0.362µs 141.860µs ± 0.200µs 142.057µs 142.335µs 142.650µs 143.258µs 0.99% -0.393 1.847 0.25% 0.026µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [141.783µs; 141.883µs] or [-0.035%; +0.035%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.691µs 2.749µs ± 0.018µs 2.752µs ± 0.010µs 2.761µs 2.775µs 2.786µs 2.794µs 1.51% -0.631 0.706 0.65% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.747µs; 2.752µs] or [-0.090%; +0.090%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.925ms 8.952ms ± 0.012ms 8.950ms ± 0.008ms 8.958ms 8.972ms 8.982ms 9.015ms 0.72% 1.149 4.042 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.950ms; 8.953ms] or [-0.019%; +0.019%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.195µs 1.372µs ± 0.054µs 1.385µs ± 0.033µs 1.413µs 1.445µs 1.468µs 1.469µs 6.04% -0.747 0.743 3.91% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.365µs; 1.380µs] or [-0.543%; +0.543%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 74.823µs 75.197µs ± 0.128µs 75.192µs ± 0.024µs 75.217µs 75.247µs 75.324µs 76.859µs 2.22% 10.923 141.429 0.17% 0.009µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [75.180µs; 75.215µs] or [-0.024%; +0.024%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 268.622µs 272.652µs ± 1.502µs 272.467µs ± 0.987µs 273.617µs 275.246µs 276.097µs 278.203µs 2.11% 0.457 0.326 0.55% 0.106µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3594496.682op/s 3667793.957op/s ± 20153.416op/s 3670169.456op/s ± 13251.280op/s 3681698.175op/s 3696672.974op/s 3705802.460op/s 3722700.045op/s 1.43% -0.423 0.273 0.55% 1425.062op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 25.800µs 25.843µs ± 0.034µs 25.844µs ± 0.030µs 25.862µs 25.903µs 25.924µs 26.002µs 0.61% 0.938 1.326 0.13% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 38458817.319op/s 38694866.048op/s ± 50129.248op/s 38694145.602op/s ± 44281.336op/s 38740682.937op/s 38750967.838op/s 38756330.134op/s 38759123.020op/s 0.17% -0.929 1.278 0.13% 3544.673op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 15.328µs 15.378µs ± 0.067µs 15.374µs ± 0.022µs 15.394µs 15.423µs 15.441µs 16.246µs 5.67% 10.737 135.715 0.44% 0.005µs 1 200
normalization/normalize_name/normalize_name/good throughput 61554319.600op/s 65029776.641op/s ± 272522.030op/s 65045454.732op/s ± 91838.769op/s 65147020.401op/s 65205775.479op/s 65225782.718op/s 65241562.414op/s 0.30% -10.442 130.659 0.42% 19270.218op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [272.444µs; 272.860µs] or [-0.076%; +0.076%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3665000.887op/s; 3670587.027op/s] or [-0.076%; +0.076%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [25.839µs; 25.848µs] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [38687918.617op/s; 38701813.480op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/good execution_time [15.369µs; 15.387µs] or [-0.061%; +0.061%] None None None
normalization/normalize_name/normalize_name/good throughput [64992007.708op/s; 65067545.573op/s] or [-0.058%; +0.058%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.208µs 1.209µs ± 0.001µs 1.208µs ± 0.000µs 1.209µs 1.210µs 1.212µs 1.215µs 0.56% 3.568 17.998 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 822934418.174op/s 827464570.271op/s ± 603985.539op/s 827566613.558op/s ± 235528.721op/s 827820127.443op/s 827932985.675op/s 827995024.762op/s 828003165.275op/s 0.05% -3.552 17.828 0.07% 42708.227op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 108.822µs 110.857µs ± 0.749µs 110.890µs ± 0.507µs 111.384µs 111.774µs 112.083µs 114.494µs 3.25% 0.421 2.971 0.67% 0.053µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 8734049.927op/s 9021013.601op/s ± 60763.830op/s 9017936.225op/s ± 41208.566op/s 9061017.801op/s 9131927.345op/s 9160440.378op/s 9189350.237op/s 1.90% -0.327 2.639 0.67% 4296.652op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 101.605µs 103.558µs ± 0.771µs 103.641µs ± 0.391µs 103.933µs 104.513µs 105.988µs 107.286µs 3.52% 0.590 3.416 0.74% 0.055µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 9320858.245op/s 9656911.462op/s ± 71621.927op/s 9648704.606op/s ± 36581.030op/s 9702233.066op/s 9768744.395op/s 9835487.552op/s 9842047.316op/s 2.00% -0.480 3.085 0.74% 5064.435op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.208µs 1.209µs ± 0.001µs 1.209µs ± 0.001µs 1.210µs 1.211µs 1.212µs 1.214µs 0.42% 1.159 0.606 0.10% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 823943558.921op/s 827114709.991op/s ± 803800.309op/s 827439563.401op/s ± 391192.470op/s 827715830.072op/s 827929137.669op/s 827962397.858op/s 828139410.771op/s 0.08% -1.156 0.592 0.10% 56837.265op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 98.263µs 100.280µs ± 0.834µs 100.272µs ± 0.420µs 100.659µs 101.906µs 102.691µs 102.918µs 2.64% 0.501 1.097 0.83% 0.059µs 1 200
credit_card/is_card_number/378282246310005 throughput 9716470.477op/s 9972772.357op/s ± 82586.058op/s 9972896.759op/s ± 41914.147op/s 10016441.014op/s 10109431.029op/s 10162203.111op/s 10176724.935op/s 2.04% -0.430 1.016 0.83% 5839.716op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 94.118µs 95.671µs ± 0.876µs 95.411µs ± 0.375µs 95.960µs 98.102µs 98.620µs 99.308µs 4.08% 1.849 4.064 0.91% 0.062µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 10069725.658op/s 10453359.719op/s ± 94125.012op/s 10480924.215op/s ± 41024.626op/s 10510386.376op/s 10555804.745op/s 10594305.440op/s 10624989.185op/s 1.37% -1.777 3.786 0.90% 6655.643op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 20.522µs 21.278µs ± 0.343µs 21.274µs ± 0.250µs 21.523µs 21.856µs 22.063µs 22.162µs 4.17% 0.109 -0.455 1.61% 0.024µs 1 200
credit_card/is_card_number/x371413321323331 throughput 45122078.214op/s 47009909.173op/s ± 756419.106op/s 47005798.709op/s ± 551783.597op/s 47566855.622op/s 48298877.804op/s 48463215.368op/s 48727700.094op/s 3.66% -0.035 -0.483 1.61% 53486.908op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.207µs 1.209µs ± 0.001µs 1.208µs ± 0.000µs 1.209µs 1.211µs 1.212µs 1.213µs 0.35% 2.151 3.702 0.09% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 824688194.714op/s 827412438.497op/s ± 743557.514op/s 827570722.951op/s ± 247909.841op/s 827871209.927op/s 828006473.322op/s 828039568.044op/s 828179592.308op/s 0.07% -2.148 3.692 0.09% 52577.456op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 93.702µs 95.894µs ± 0.655µs 95.957µs ± 0.390µs 96.296µs 96.781µs 97.409µs 97.604µs 1.72% -0.585 1.046 0.68% 0.046µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 10245516.733op/s 10428661.661op/s ± 71578.524op/s 10421376.480op/s ± 42450.324op/s 10467625.810op/s 10566386.935op/s 10633784.522op/s 10672164.424op/s 2.41% 0.641 1.127 0.68% 5061.366op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 84.435µs 86.952µs ± 0.687µs 87.057µs ± 0.324µs 87.332µs 87.848µs 88.716µs 88.906µs 2.12% -0.676 1.716 0.79% 0.049µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 11247796.620op/s 11501314.054op/s ± 91434.685op/s 11486790.785op/s ± 42792.110op/s 11540715.346op/s 11665782.864op/s 11751575.798op/s 11843386.326op/s 3.10% 0.753 1.825 0.79% 6465.409op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.208µs 1.209µs ± 0.002µs 1.209µs ± 0.001µs 1.211µs 1.212µs 1.212µs 1.220µs 0.91% 1.971 8.243 0.13% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 819960301.040op/s 826951089.021op/s ± 1052106.599op/s 827412542.498op/s ± 435650.452op/s 827704484.396op/s 827919999.677op/s 827989554.620op/s 827999697.818op/s 0.07% -1.947 8.019 0.13% 74395.171op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 81.170µs 83.004µs ± 0.686µs 82.990µs ± 0.378µs 83.353µs 83.987µs 85.222µs 85.781µs 3.36% 0.607 1.790 0.82% 0.049µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 11657654.440op/s 12048409.764op/s ± 99111.341op/s 12049655.842op/s ± 54858.342op/s 12107106.421op/s 12201680.845op/s 12256655.078op/s 12319759.147op/s 2.24% -0.524 1.601 0.82% 7008.230op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 94.225µs 95.621µs ± 0.748µs 95.449µs ± 0.302µs 95.877µs 96.848µs 98.582µs 99.431µs 4.17% 2.048 6.581 0.78% 0.053µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 10057234.284op/s 10458573.700op/s ± 80545.377op/s 10476794.964op/s ± 33192.153op/s 10505278.955op/s 10553348.825op/s 10572315.023op/s 10612911.830op/s 1.30% -1.948 6.024 0.77% 5695.418op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 20.475µs 21.279µs ± 0.364µs 21.278µs ± 0.241µs 21.527µs 21.901µs 22.068µs 22.430µs 5.41% 0.194 -0.232 1.71% 0.026µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 44583861.100op/s 47007418.027op/s ± 802893.114op/s 46996040.193op/s ± 538528.158op/s 47533978.120op/s 48349528.360op/s 48680048.990op/s 48839783.803op/s 3.92% -0.106 -0.295 1.70% 56773.117op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.208µs; 1.209µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [827380863.684op/s; 827548276.858op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [110.754µs; 110.961µs] or [-0.094%; +0.094%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [9012592.318op/s; 9029434.883op/s] or [-0.093%; +0.093%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [103.452µs; 103.665µs] or [-0.103%; +0.103%] None None None
credit_card/is_card_number/ 378282246310005 throughput [9646985.351op/s; 9666837.572op/s] or [-0.103%; +0.103%] None None None
credit_card/is_card_number/37828224631 execution_time [1.209µs; 1.209µs] or [-0.013%; +0.013%] None None None
credit_card/is_card_number/37828224631 throughput [827003310.999op/s; 827226108.983op/s] or [-0.013%; +0.013%] None None None
credit_card/is_card_number/378282246310005 execution_time [100.164µs; 100.395µs] or [-0.115%; +0.115%] None None None
credit_card/is_card_number/378282246310005 throughput [9961326.723op/s; 9984217.990op/s] or [-0.115%; +0.115%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [95.550µs; 95.792µs] or [-0.127%; +0.127%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [10440314.897op/s; 10466404.540op/s] or [-0.125%; +0.125%] None None None
credit_card/is_card_number/x371413321323331 execution_time [21.230µs; 21.325µs] or [-0.223%; +0.223%] None None None
credit_card/is_card_number/x371413321323331 throughput [46905076.759op/s; 47114741.586op/s] or [-0.223%; +0.223%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.208µs; 1.209µs] or [-0.012%; +0.012%] None None None
credit_card/is_card_number_no_luhn/ throughput [827309388.577op/s; 827515488.417op/s] or [-0.012%; +0.012%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [95.803µs; 95.985µs] or [-0.095%; +0.095%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [10418741.566op/s; 10438581.756op/s] or [-0.095%; +0.095%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [86.857µs; 87.047µs] or [-0.110%; +0.110%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [11488642.086op/s; 11513986.022op/s] or [-0.110%; +0.110%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.209µs; 1.209µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [826805277.165op/s; 827096900.877op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [82.909µs; 83.099µs] or [-0.115%; +0.115%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12034673.885op/s; 12062145.643op/s] or [-0.114%; +0.114%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [95.517µs; 95.725µs] or [-0.108%; +0.108%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [10447410.885op/s; 10469736.515op/s] or [-0.107%; +0.107%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [21.229µs; 21.330µs] or [-0.237%; +0.237%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [46896144.763op/s; 47118691.290op/s] or [-0.237%; +0.237%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 293.282ns 303.615ns ± 11.410ns 298.760ns ± 3.854ns 307.567ns 334.813ns 336.457ns 336.743ns 12.71% 1.711 2.089 3.75% 0.807ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [302.033ns; 305.196ns] or [-0.521%; +0.521%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 37.471µs 37.953µs ± 0.866µs 37.552µs ± 0.054µs 37.660µs 39.791µs 39.830µs 40.919µs 8.96% 1.723 1.143 2.28% 0.061µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [37.833µs; 38.073µs] or [-0.316%; +0.316%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 5532b66 1726597797 andrew.glaude/dogstatsd-client
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.335µs 3.135µs ± 1.531µs 2.975µs ± 0.022µs 2.997µs 3.088µs 13.600µs 17.590µs 491.28% 7.991 64.503 48.72% 0.108µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.923µs; 3.347µs] or [-6.769%; +6.769%] None None None

Baseline

Omitted due to size.

@bwoebi bwoebi force-pushed the andrew.glaude/dogstatsd-client branch from ae5c14e to 8fecb9a Compare September 17, 2024 23:14
@ajgajg1134
Copy link
Contributor Author

Closing this PR in favor of this other branch where I split the implementation which is much simpler to read (at the cost of some code duplication) https://github.com/DataDog/libdatadog/pull/638/files

@ajgajg1134 ajgajg1134 closed this Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants