Skip to content

Commit

Permalink
Merge branch 'master' into base-merge
Browse files Browse the repository at this point in the history
* master: (98 commits)
  DEBUG-2334 Remove old ddtrace require from tracing benchmarks (DataDog#3783)
  Avoid environment starts with number
  Add more variables for artifacts unpack
  `git fetch` before rev-parse
  Use ruby image that has `git` installed
  Provide versions
  Fixed: RSpec #log_deprecation matcher error when log is limited.
  Remove internal entry from changelog
  Update lockfiles for release 2.2.0
  Bump version 2.1.0 to 2.2.0
  Add 2.2.0 to CHANGELOG.md
  Add VSCode extension recommendations
  Add `ruby-lsp` gem
  [PROF-3760] Revert "Merge pull request DataDog#3760 from DataDog/ivoanjo/prof-10123-placeholder-missing-allocations"
  [PROF-10112] Also simplify actionview templates with three underscores
  remove duplicated comments
  Use ActiveSupport events to instrument ActiveSupport::Cache
  [PROF-10125] Track unscaled allocation counts in allocation profiler
  Bootstrap new profile value type for unscaled counts
  apply static analysis proposed fixes
  ...
  • Loading branch information
p committed Jul 16, 2024
2 parents e51151f + 161e3eb commit 2e50f4f
Show file tree
Hide file tree
Showing 506 changed files with 5,177 additions and 2,151 deletions.
22 changes: 22 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -494,6 +494,11 @@ job_configuration:
ruby_version: 'jruby-9.3.9.0'
image: ghcr.io/datadog/dd-trace-rb/jruby:9.3.9.0-dd
resource_class_to_use: large
- &config-jruby-9_4
<<: *filters_all_branches_and_tags
ruby_version: 'jruby-9.4.7.0'
image: ghcr.io/datadog/dd-trace-rb/jruby:9.4.7.0-dd
resource_class_to_use: large

workflows:
version: 2
Expand All @@ -518,6 +523,7 @@ workflows:
# ADD NEW RUBIES HERE
- test-jruby-9.2
- test-jruby-9.3
- test-jruby-9.4
- orb/changelog:
<<: *config-2_7-small
name: changelog
Expand Down Expand Up @@ -638,6 +644,14 @@ workflows:
name: test-jruby-9.3
requires:
- build-jruby-9.3
- orb/build:
<<: *config-jruby-9_4
name: build-jruby-9.4
- orb/test:
<<: *config-jruby-9_4
name: test-jruby-9.4
requires:
- build-jruby-9.4
# This workflow runs the same `build` and `test` jobs as above on a schedule.
# Tasks related to housekeeping (e.g. prerelease) are not relevant
# to this daily check, as they are not expected to be impacted here.
Expand Down Expand Up @@ -769,3 +783,11 @@ workflows:
name: test-jruby-9.3
requires:
- build-jruby-9.3
- orb/build:
<<: *config-jruby-9_4
name: build-jruby-9.4
- orb/test:
<<: *config-jruby-9_4
name: test-jruby-9.4
requires:
- build-jruby-9.4
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: Bug report
name: 🐛 Bug report
about: File a bug report
title: ''
labels: community, bug
Expand Down
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: ℹ️ Datadog Support
url: https://www.datadoghq.com/support/
about: Get help from the Datadog support team
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: Feature request
name: 💡 Feature request
about: Suggest an idea for this project
title: ''
labels: community, feature-request
Expand Down
25 changes: 25 additions & 0 deletions .github/workflows/datadog-sca.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
on: [push]

name: Datadog Software Composition Analysis

jobs:
software-composition-analysis:
runs-on: ubuntu-latest
name: Datadog SBOM Generation and Upload
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
bundler-cache: true
ruby-version: "3.3"
- name: Check imported libraries are secure and compliant
id: datadog-software-composition-analysis
uses: DataDog/datadog-sca-github-action@main
with:
dd_api_key: ${{ secrets.DD_API_KEY }}
dd_app_key: ${{ secrets.DD_APP_KEY }}
dd_service: dd-trace-rb
dd_env: ci
dd_site: datadoghq.com
21 changes: 21 additions & 0 deletions .github/workflows/datadog-static-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
on: [push]

name: Datadog Static Analysis

jobs:
static-analysis:
runs-on: ubuntu-latest
name: Datadog Static Analyzer
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Check code meets quality and security standards
id: datadog-static-analysis
uses: DataDog/datadog-static-analyzer-github-action@v1
with:
dd_api_key: ${{ secrets.DD_API_KEY }}
dd_app_key: ${{ secrets.DD_APP_KEY }}
dd_service: dd-trace-rb
dd_env: ci
dd_site: datadoghq.com
cpu_count: 2
4 changes: 2 additions & 2 deletions .github/workflows/system-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -302,10 +302,10 @@ jobs:
if: ${{ always() }}
name: Aggregate (${{ matrix.app }})
steps:
- name: Setup python 3.9
- name: Setup python 3.12
uses: actions/setup-python@v5
with:
python-version: '3.9'
python-version: '3.12'
- name: Checkout
uses: actions/checkout@v4
with:
Expand Down
23 changes: 22 additions & 1 deletion .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -232,9 +232,27 @@ onboarding_tests:
variables:
PRODUCT_NAME: auto_inject-ruby

save_versions:
image: $DOCKER_REGISTRY/images/mirror/ruby:3.2.2
stage: deploy
needs: []
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: on_success
- when: manual
allow_failure: true
script:
- git fetch --all
- echo COMMIT_SHA_1X=$(git rev-parse origin/1.x-stable) >> upstream.env
- echo COMMIT_SHA_2X=$(git rev-parse origin/master) >> upstream.env
artifacts:
paths:
- upstream.env

deploy_to_reliability_env:
stage: deploy
needs: [] # This allows the job to run without prerequisites
needs:
- save_versions
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: on_success
Expand All @@ -248,6 +266,9 @@ deploy_to_reliability_env:
UPSTREAM_PROJECT_ID: $CI_PROJECT_ID
UPSTREAM_PROJECT_NAME: $CI_PROJECT_NAME
UPSTREAM_COMMIT_SHA: $CI_COMMIT_SHA
UPSTREAM_PACKAGE_JOB: save_versions
UPSTREAM_PIPELINE_ID: $CI_PIPELINE_ID
CI_SERVER_URL: $CI_SERVER_URL

prepare_image_destinations:
stage: deploy
Expand Down
8 changes: 8 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"recommendations": [
"shopify.ruby-lsp",
"editorconfig.editorconfig",
"soutaro.rbs-syntax",
"soutaro.steep-vscode"
]
}
3 changes: 3 additions & 0 deletions Appraisals
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ REMOVED_GEMS = {
'rbs',
'steep',
],
:dev => [
'ruby-lsp',
],
}

def appraise(group, &block)
Expand Down
55 changes: 53 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,36 @@

## [Unreleased]

## [2.2.0] - 2024-07-11

### Added

* Tracing: Add `Rails` Runner instrumentation ([#2509][])
* Tracing: Introduce a new, reworked `GraphQL` tracer to comply with span attributes specification ([#3672][])
* Tracing: Enhance `ActiveSupport::Cache` instrumentation with `ActiveSupport::Notifications` subscription ([#3772][])
* Profiling: Track unscaled allocation counts in allocation profiler ([#3770][])

### Changed

* Core: Send Telemetry events in batches ([#3749][])
* Tracing: Populate spans from `ActiveSupport::Notifications` as early as possible ([#3725][])
* Profiling: Upgrade to `libdatadog` 10 ([#3753][])
* Profiling: Optimize `CodeProvenance#record_loaded_files` to avoid allocations ([#3757][])

### Fixed

* Core: Fix Telemetry events blocking main thread ([#3718][])
* Core: Fix deadlock from Telemetry threads ([#3743][])
* Tracing: Fix empty log correlation when tracing is disabled ([#3731][])
* Tracing: Fix HTTP propagation when missing parent span id ([#3730][])
* Tracing: Ensure `_dd.p.tid` tag with fixed size ([#3729][])
* OTel: Fix ids encoding/decoding for propagation ([#3709][])
* Profiling: Workaround Ruby `Dir` returning incorrect results ([#3720][])
* Profiling: Fix `Phusion Passenger` detection when missing from `Gemfile`/`gems.rb` ([#3750][])
* Profiling: Fix `rpath` for linking to libdatadog when loading extension ([#3706][])
* Profiling: Fix incorrect code provenance due to broken JSON monkey patch ([#3695][])
* Profiling: Fix aggregation of actionview template classes ([#3759][], [#3774][])

## [2.1.0] - 2024-06-10

### Added
Expand Down Expand Up @@ -2904,7 +2934,8 @@ Release notes: https://github.com/DataDog/dd-trace-rb/releases/tag/v0.3.1
Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1


[Unreleased]: https://github.com/DataDog/dd-trace-rb/compare/v2.1.0...master
[Unreleased]: https://github.com/DataDog/dd-trace-rb/compare/v2.2.0...master
[2.2.0]: https://github.com/DataDog/dd-trace-rb/compare/v2.1.0...v2.2.0
[2.1.0]: https://github.com/DataDog/dd-trace-rb/compare/v2.0.0...v2.1.0
[2.0.0]: https://github.com/DataDog/dd-trace-rb/compare/v2.0.0.rc1...v2.0.0
[2.0.0.rc1]: https://github.com/DataDog/dd-trace-rb/compare/v2.0.0.beta2...v2.0.0.rc1
Expand Down Expand Up @@ -3901,6 +3932,7 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
[#2497]: https://github.com/DataDog/dd-trace-rb/issues/2497
[#2501]: https://github.com/DataDog/dd-trace-rb/issues/2501
[#2504]: https://github.com/DataDog/dd-trace-rb/issues/2504
[#2509]: https://github.com/DataDog/dd-trace-rb/issues/2509
[#2512]: https://github.com/DataDog/dd-trace-rb/issues/2512
[#2513]: https://github.com/DataDog/dd-trace-rb/issues/2513
[#2522]: https://github.com/DataDog/dd-trace-rb/issues/2522
Expand Down Expand Up @@ -4283,6 +4315,25 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
[#3651]: https://github.com/DataDog/dd-trace-rb/issues/3651
[#3657]: https://github.com/DataDog/dd-trace-rb/issues/3657
[#3664]: https://github.com/DataDog/dd-trace-rb/issues/3664
[#3672]: https://github.com/DataDog/dd-trace-rb/issues/3672
[#3695]: https://github.com/DataDog/dd-trace-rb/issues/3695
[#3706]: https://github.com/DataDog/dd-trace-rb/issues/3706
[#3709]: https://github.com/DataDog/dd-trace-rb/issues/3709
[#3718]: https://github.com/DataDog/dd-trace-rb/issues/3718
[#3720]: https://github.com/DataDog/dd-trace-rb/issues/3720
[#3725]: https://github.com/DataDog/dd-trace-rb/issues/3725
[#3729]: https://github.com/DataDog/dd-trace-rb/issues/3729
[#3730]: https://github.com/DataDog/dd-trace-rb/issues/3730
[#3731]: https://github.com/DataDog/dd-trace-rb/issues/3731
[#3743]: https://github.com/DataDog/dd-trace-rb/issues/3743
[#3749]: https://github.com/DataDog/dd-trace-rb/issues/3749
[#3750]: https://github.com/DataDog/dd-trace-rb/issues/3750
[#3753]: https://github.com/DataDog/dd-trace-rb/issues/3753
[#3757]: https://github.com/DataDog/dd-trace-rb/issues/3757
[#3759]: https://github.com/DataDog/dd-trace-rb/issues/3759
[#3770]: https://github.com/DataDog/dd-trace-rb/issues/3770
[#3772]: https://github.com/DataDog/dd-trace-rb/issues/3772
[#3774]: https://github.com/DataDog/dd-trace-rb/issues/3774
[@AdrianLC]: https://github.com/AdrianLC
[@Azure7111]: https://github.com/Azure7111
[@BabyGroot]: https://github.com/BabyGroot
Expand Down Expand Up @@ -4434,4 +4485,4 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
[@y-yagi]: https://github.com/y-yagi
[@yujideveloper]: https://github.com/yujideveloper
[@yukimurasawa]: https://github.com/yukimurasawa
[@zachmccormick]: https://github.com/zachmccormick
[@zachmccormick]: https://github.com/zachmccormick
4 changes: 4 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,10 @@ group :check do
end
end

group :dev do
gem 'ruby-lsp', require: false if RUBY_VERSION >= '3.0.0' && RUBY_PLATFORM != 'java'
end

# `1.17.0` provides broken RBS type definitions
# https://github.com/ffi/ffi/blob/master/CHANGELOG.md#1170rc1--2024-04-08
#
Expand Down
7 changes: 4 additions & 3 deletions Matrixfile
Original file line number Diff line number Diff line change
Expand Up @@ -211,9 +211,10 @@
'rails61-semantic-logger' => '✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ✅ 3.4 / ✅ jruby'
},
'action_cable' => {
'rails5-mysql2' => '✅ 2.5 / ✅ 2.6 / ❌ 2.7 / ❌ 3.0 / ❌ 3.1 / ❌ 3.2 / ❌ 3.3 / ❌ 3.4 / ✅ jruby',
'rails6-mysql2' => '✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ❌ 3.0 / ❌ 3.1 / ❌ 3.2 / ❌ 3.3 / ❌ 3.4 / ✅ jruby',
'rails61-mysql2' => '✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ✅ 3.4 / ✅ jruby'
# FIXME: Enable the test for JRuby after fixing `log writing failed. closed stream` in CircleCI.
'rails5-mysql2' => '✅ 2.5 / ✅ 2.6 / ❌ 2.7 / ❌ 3.0 / ❌ 3.1 / ❌ 3.2 / ❌ 3.3 / ❌ 3.4 / ❌ jruby',
'rails6-mysql2' => '✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ❌ 3.0 / ❌ 3.1 / ❌ 3.2 / ❌ 3.3 / ❌ 3.4 / ❌ jruby',
'rails61-mysql2' => '✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ✅ 3.4 / ❌ jruby'
},
'action_mailer' => {
'rails5-mysql2' => '✅ 2.5 / ✅ 2.6 / ❌ 2.7 / ❌ 3.0 / ❌ 3.1 / ❌ 3.2 / ❌ 3.3 / ❌ 3.4 / ✅ jruby',
Expand Down
10 changes: 4 additions & 6 deletions appraisal/jruby-9.4.rb
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
# TODO: Actually test JRuby 9.4

appraise 'rails61-mysql2' do
gem 'rails', '~> 6.1.0'
gem 'activerecord-jdbcmysql-adapter', platform: :jruby
gem 'activerecord-jdbcmysql-adapter', '~> 61.0', platform: :jruby
gem 'sprockets', '< 4'
gem 'lograge', '~> 0.11'
gem 'net-smtp'
Expand Down Expand Up @@ -82,12 +80,12 @@
end

appraise 'relational_db' do
gem 'activerecord', '~> 7'
gem 'activerecord', '~> 6.1.0'
gem 'delayed_job'
gem 'delayed_job_active_record'
gem 'makara', '>= 0.6.0.pre' # Ruby 3 requires >= 0.6.0, which is currently in pre-release: https://rubygems.org/gems/makara/versions
gem 'activerecord-jdbcmysql-adapter', platform: :jruby
gem 'activerecord-jdbcpostgresql-adapter', platform: :jruby
gem 'activerecord-jdbcmysql-adapter', '~> 61.0', platform: :jruby
gem 'activerecord-jdbcpostgresql-adapter', '~> 61.0', platform: :jruby
gem 'sequel', '~> 5.54.0' # TODO: Support sequel 5.62.0+
gem 'jdbc-sqlite3', '>= 3.28', platform: :jruby
end
Expand Down
44 changes: 44 additions & 0 deletions benchmarks/profiler_hold_resume_interruptions.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Used to quickly run benchmark under RSpec as part of the usual test suite, to validate it didn't bitrot
VALIDATE_BENCHMARK_MODE = ENV['VALIDATE_BENCHMARK'] == 'true'

return unless __FILE__ == $PROGRAM_NAME || VALIDATE_BENCHMARK_MODE

require 'benchmark/ips'
require 'datadog'
require 'pry'
require_relative 'dogstatsd_reporter'

# This benchmark measures the performance of the hold/resume interruptions used by the DirMonkeyPatches
class ProfilerHoldResumeInterruptions
def create_profiler
Datadog.configure do |c|
c.profiling.enabled = true
end
Datadog::Profiling.wait_until_running
end

def run_benchmark
Benchmark.ips do |x|
benchmark_time = VALIDATE_BENCHMARK_MODE ? { time: 0.01, warmup: 0 } : { time: 10, warmup: 2 }
x.config(
**benchmark_time,
suite: report_to_dogstatsd_if_enabled_via_environment_variable(benchmark_name: 'profiler_hold_resume_interruptions')
)

x.report("hold / resume") do
Datadog::Profiling::Collectors::CpuAndWallTimeWorker._native_hold_signals
Datadog::Profiling::Collectors::CpuAndWallTimeWorker._native_resume_signals
end

x.save! 'profiler_hold_resume_interruptions-results.json' unless VALIDATE_BENCHMARK_MODE
x.compare!
end
end
end

puts "Current pid is #{Process.pid}"

ProfilerHoldResumeInterruptions.new.instance_exec do
create_profiler
run_benchmark
end
8 changes: 1 addition & 7 deletions benchmarks/tracing_trace.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,7 @@

require 'benchmark/ips'
require 'open3'

begin
require 'datadog'
rescue LoadError # TODO: Remove when ddtrace 2.0 is merged to master
# This is required to run benchmarks against ddtrace 1.x.
require 'ddtrace'
end
require 'datadog'

class TracingTraceBenchmark
module NoopWriter
Expand Down
2 changes: 1 addition & 1 deletion datadog.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Gem::Specification.new do |spec|
# Used by profiling (and possibly others in the future)
# When updating the version here, please also update the version in `native_extension_helpers.rb`
# (and yes we have a test for it)
spec.add_dependency 'libdatadog', '~> 9.0.0.1.0'
spec.add_dependency 'libdatadog', '~> 10.0.0.1.0'

spec.extensions = ['ext/datadog_profiling_native_extension/extconf.rb', 'ext/datadog_profiling_loader/extconf.rb']
end
Loading

0 comments on commit 2e50f4f

Please sign in to comment.