Add `apm-sdks-benchmarks` to CI #10206

sarahchen6 · 2025-12-12T22:48:10Z

What Does This Do

Test https://github.com/DataDog/apm-sdks-benchmarks/pull/44

and https://github.com/DataDog/apm-sdks-benchmarks/pull/45

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-12-13T00:47:30Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/test-merge-base-benchmark
git_commit_date	1767714789	1767884162
git_commit_sha	`3917d52`	`9da26aa`
release_version	1.59.0-SNAPSHOT~3917d52aab	1.57.0-SNAPSHOT~9da26aafd7

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1767886244	1767886244
ci_job_id	1336196538	1336196538
ci_pipeline_id	89677605	89677605
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-7qntkrgx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-7qntkrgx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 9 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.082 s) : 0, 1082301
Total [baseline] (10.834 s) : 0, 10834093
Agent [candidate] (1.084 s) : 0, 1083559
Total [candidate] (10.817 s) : 0, 10817417
section appsec
Agent [baseline] (1.278 s) : 0, 1278147
Total [baseline] (10.965 s) : 0, 10964804
Agent [candidate] (1.271 s) : 0, 1271038
Total [candidate] (10.977 s) : 0, 10977310
section iast
Agent [baseline] (1.223 s) : 0, 1222676
Total [baseline] (11.114 s) : 0, 11114484
Agent [candidate] (1.232 s) : 0, 1231793
Total [candidate] (11.155 s) : 0, 11154584
section profiling
Agent [baseline] (1.205 s) : 0, 1204801
Total [baseline] (10.917 s) : 0, 10917431
Agent [candidate] (1.211 s) : 0, 1211129
Total [candidate] (10.941 s) : 0, 10940501

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.082 s	-
Agent	appsec	1.278 s	195.846 ms (18.1%)
Agent	iast	1.223 s	140.375 ms (13.0%)
Agent	profiling	1.205 s	122.5 ms (11.3%)
Total	tracing	10.834 s	-
Total	appsec	10.965 s	130.711 ms (1.2%)
Total	iast	11.114 s	280.391 ms (2.6%)
Total	profiling	10.917 s	83.338 ms (0.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.084 s	-
Agent	appsec	1.271 s	187.479 ms (17.3%)
Agent	iast	1.232 s	148.235 ms (13.7%)
Agent	profiling	1.211 s	127.571 ms (11.8%)
Total	tracing	10.817 s	-
Total	appsec	10.977 s	159.893 ms (1.5%)
Total	iast	11.155 s	337.166 ms (3.1%)
Total	profiling	10.941 s	123.084 ms (1.1%)

gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.176 ms) : 0, 1176
crashtracking [candidate] (1.18 ms) : 0, 1180
BytebuddyAgent [baseline] (649.32 ms) : 0, 649320
BytebuddyAgent [candidate] (650.752 ms) : 0, 650752
GlobalTracer [baseline] (282.374 ms) : 0, 282374
GlobalTracer [candidate] (282.129 ms) : 0, 282129
AppSec [baseline] (32.678 ms) : 0, 32678
AppSec [candidate] (32.565 ms) : 0, 32565
Debugger [baseline] (67.936 ms) : 0, 67936
Debugger [candidate] (68.109 ms) : 0, 68109
Remote Config [baseline] (615.842 µs) : 0, 616
Remote Config [candidate] (630.145 µs) : 0, 630
Telemetry [baseline] (8.996 ms) : 0, 8996
Telemetry [candidate] (8.985 ms) : 0, 8985
Flare Poller [baseline] (3.728 ms) : 0, 3728
Flare Poller [candidate] (3.742 ms) : 0, 3742
section appsec
crashtracking [baseline] (1.192 ms) : 0, 1192
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (700.005 ms) : 0, 700005
BytebuddyAgent [candidate] (695.536 ms) : 0, 695536
GlobalTracer [baseline] (261.027 ms) : 0, 261027
GlobalTracer [candidate] (259.738 ms) : 0, 259738
AppSec [baseline] (174.799 ms) : 0, 174799
AppSec [candidate] (173.867 ms) : 0, 173867
Debugger [baseline] (66.564 ms) : 0, 66564
Debugger [candidate] (66.568 ms) : 0, 66568
Remote Config [baseline] (759.241 µs) : 0, 759
Remote Config [candidate] (759.149 µs) : 0, 759
Telemetry [baseline] (9.268 ms) : 0, 9268
Telemetry [candidate] (9.333 ms) : 0, 9333
Flare Poller [baseline] (3.759 ms) : 0, 3759
Flare Poller [candidate] (3.731 ms) : 0, 3731
IAST [baseline] (25.142 ms) : 0, 25142
IAST [candidate] (24.768 ms) : 0, 24768
section iast
crashtracking [baseline] (1.176 ms) : 0, 1176
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (790.154 ms) : 0, 790154
BytebuddyAgent [candidate] (797.688 ms) : 0, 797688
GlobalTracer [baseline] (256.149 ms) : 0, 256149
GlobalTracer [candidate] (257.398 ms) : 0, 257398
AppSec [baseline] (32.851 ms) : 0, 32851
AppSec [candidate] (33.612 ms) : 0, 33612
Debugger [baseline] (67.379 ms) : 0, 67379
Debugger [candidate] (66.479 ms) : 0, 66479
Remote Config [baseline] (568.641 µs) : 0, 569
Remote Config [candidate] (590.645 µs) : 0, 591
Telemetry [baseline] (8.407 ms) : 0, 8407
Telemetry [candidate] (8.515 ms) : 0, 8515
Flare Poller [baseline] (3.583 ms) : 0, 3583
Flare Poller [candidate] (3.615 ms) : 0, 3615
IAST [baseline] (27.119 ms) : 0, 27119
IAST [candidate] (27.185 ms) : 0, 27185
section profiling
crashtracking [baseline] (1.213 ms) : 0, 1213
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (703.375 ms) : 0, 703375
BytebuddyAgent [candidate] (707.35 ms) : 0, 707350
GlobalTracer [baseline] (220.266 ms) : 0, 220266
GlobalTracer [candidate] (221.752 ms) : 0, 221752
AppSec [baseline] (32.138 ms) : 0, 32138
AppSec [candidate] (32.403 ms) : 0, 32403
Debugger [baseline] (68.281 ms) : 0, 68281
Debugger [candidate] (68.596 ms) : 0, 68596
Remote Config [baseline] (631.238 µs) : 0, 631
Remote Config [candidate] (616.224 µs) : 0, 616
Telemetry [baseline] (8.746 ms) : 0, 8746
Telemetry [candidate] (8.729 ms) : 0, 8729
Flare Poller [baseline] (3.669 ms) : 0, 3669
Flare Poller [candidate] (3.678 ms) : 0, 3678
ProfilingAgent [baseline] (96.758 ms) : 0, 96758
ProfilingAgent [candidate] (96.639 ms) : 0, 96639
Profiling [baseline] (97.334 ms) : 0, 97334
Profiling [candidate] (97.218 ms) : 0, 97218

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.089 s) : 0, 1089005
Total [baseline] (8.785 s) : 0, 8785214
Agent [candidate] (1.083 s) : 0, 1083088
Total [candidate] (8.742 s) : 0, 8741885
section iast
Agent [baseline] (1.226 s) : 0, 1226432
Total [baseline] (9.31 s) : 0, 9309948
Agent [candidate] (1.222 s) : 0, 1222092
Total [candidate] (9.326 s) : 0, 9326426

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.089 s	-
Agent	iast	1.226 s	137.426 ms (12.6%)
Total	tracing	8.785 s	-
Total	iast	9.31 s	524.734 ms (6.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.083 s	-
Agent	iast	1.222 s	139.004 ms (12.8%)
Total	tracing	8.742 s	-
Total	iast	9.326 s	584.541 ms (6.7%)

gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.209 ms) : 0, 1209
BytebuddyAgent [baseline] (654.978 ms) : 0, 654978
BytebuddyAgent [candidate] (650.902 ms) : 0, 650902
GlobalTracer [baseline] (283.019 ms) : 0, 283019
GlobalTracer [candidate] (282.352 ms) : 0, 282352
AppSec [baseline] (32.795 ms) : 0, 32795
AppSec [candidate] (32.487 ms) : 0, 32487
Debugger [baseline] (67.876 ms) : 0, 67876
Debugger [candidate] (67.267 ms) : 0, 67267
Remote Config [baseline] (653.281 µs) : 0, 653
Remote Config [candidate] (627.73 µs) : 0, 628
Telemetry [baseline] (9.028 ms) : 0, 9028
Telemetry [candidate] (8.921 ms) : 0, 8921
Flare Poller [baseline] (3.792 ms) : 0, 3792
Flare Poller [candidate] (3.725 ms) : 0, 3725
section iast
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.178 ms) : 0, 1178
BytebuddyAgent [baseline] (794.342 ms) : 0, 794342
BytebuddyAgent [candidate] (790.162 ms) : 0, 790162
GlobalTracer [baseline] (256.546 ms) : 0, 256546
GlobalTracer [candidate] (255.674 ms) : 0, 255674
AppSec [baseline] (33.534 ms) : 0, 33534
AppSec [candidate] (34.512 ms) : 0, 34512
Debugger [baseline] (65.558 ms) : 0, 65558
Debugger [candidate] (65.278 ms) : 0, 65278
Remote Config [baseline] (541.975 µs) : 0, 542
Remote Config [candidate] (640.147 µs) : 0, 640
Telemetry [baseline] (8.439 ms) : 0, 8439
Telemetry [candidate] (8.577 ms) : 0, 8577
Flare Poller [baseline] (3.558 ms) : 0, 3558
Flare Poller [candidate] (3.647 ms) : 0, 3647
IAST [baseline] (27.179 ms) : 0, 27179
IAST [candidate] (26.984 ms) : 0, 26984

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/test-merge-base-benchmark
git_commit_date	1767714789	1767884162
git_commit_sha	`3917d52`	`9da26aa`
release_version	1.59.0-SNAPSHOT~3917d52aab	1.57.0-SNAPSHOT~9da26aafd7

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1767886755	1767886755
ci_job_id	1336196540	1336196540
ci_pipeline_id	89677605	89677605
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-fj81g7mq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-fj81g7mq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 16 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+144.403µs; +218.464µs] or [+5.310%; +8.033%]	worse [+242.006µs; +653.808µs] or [+3.138%; +8.477%]	unstable [-204.558op/s; +41.996op/s] or [-15.430%; +3.168%]	2.901ms	8.161ms	1244.438op/s	2.720ms	7.713ms	1325.719op/s
scenario:load:petclinic:iast:high_load	unsure [+84.319µs; +1036.803µs] or [+0.490%; +6.026%]	worse [+0.839ms; +1.788ms] or [+3.016%; +6.425%]	unstable [-41.641op/s; +18.203op/s] or [-15.527%; +6.787%]	17.767ms	29.137ms	256.469op/s	17.207ms	27.824ms	268.188op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.125 ms) : 18933, 19318
.   : milestone, 19125,
appsec (18.772 ms) : 18586, 18959
.   : milestone, 18772,
code_origins (17.867 ms) : 17689, 18046
.   : milestone, 17867,
iast (17.396 ms) : 17225, 17566
.   : milestone, 17396,
profiling (18.533 ms) : 18349, 18718
.   : milestone, 18533,
tracing (17.93 ms) : 17750, 18110
.   : milestone, 17930,
section candidate
no_agent (18.238 ms) : 18052, 18424
.   : milestone, 18238,
appsec (19.179 ms) : 18983, 19374
.   : milestone, 19179,
code_origins (17.734 ms) : 17559, 17909
.   : milestone, 17734,
iast (18.195 ms) : 18014, 18376
.   : milestone, 18195,
profiling (18.84 ms) : 18652, 19029
.   : milestone, 18840,
tracing (17.64 ms) : 17468, 17812
.   : milestone, 17640,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.125 ms [18.933 ms, 19.318 ms]	-
appsec	18.772 ms [18.586 ms, 18.959 ms]	-353.105 µs (-1.8%)
code_origins	17.867 ms [17.689 ms, 18.046 ms]	-1.258 ms (-6.6%)
iast	17.396 ms [17.225 ms, 17.566 ms]	-1.73 ms (-9.0%)
profiling	18.533 ms [18.349 ms, 18.718 ms]	-592.289 µs (-3.1%)
tracing	17.93 ms [17.75 ms, 18.11 ms]	-1.195 ms (-6.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.238 ms [18.052 ms, 18.424 ms]	-
appsec	19.179 ms [18.983 ms, 19.374 ms]	940.213 µs (5.2%)
code_origins	17.734 ms [17.559 ms, 17.909 ms]	-504.521 µs (-2.8%)
iast	18.195 ms [18.014 ms, 18.376 ms]	-42.986 µs (-0.2%)
profiling	18.84 ms [18.652 ms, 19.029 ms]	602.072 µs (3.3%)
tracing	17.64 ms [17.468 ms, 17.812 ms]	-598.266 µs (-3.3%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.179 ms) : 1167, 1191
.   : milestone, 1179,
iast (3.193 ms) : 3151, 3235
.   : milestone, 3193,
iast_FULL (5.68 ms) : 5623, 5736
.   : milestone, 5680,
iast_GLOBAL (3.458 ms) : 3409, 3508
.   : milestone, 3458,
profiling (2.178 ms) : 2156, 2200
.   : milestone, 2178,
tracing (1.82 ms) : 1805, 1836
.   : milestone, 1820,
section candidate
no_agent (1.19 ms) : 1178, 1202
.   : milestone, 1190,
iast (3.158 ms) : 3119, 3198
.   : milestone, 3158,
iast_FULL (5.833 ms) : 5774, 5891
.   : milestone, 5833,
iast_GLOBAL (3.688 ms) : 3627, 3749
.   : milestone, 3688,
profiling (2.133 ms) : 2113, 2153
.   : milestone, 2133,
tracing (1.785 ms) : 1770, 1800
.   : milestone, 1785,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.179 ms [1.167 ms, 1.191 ms]	-
iast	3.193 ms [3.151 ms, 3.235 ms]	2.014 ms (170.8%)
iast_FULL	5.68 ms [5.623 ms, 5.736 ms]	4.501 ms (381.7%)
iast_GLOBAL	3.458 ms [3.409 ms, 3.508 ms]	2.279 ms (193.3%)
profiling	2.178 ms [2.156 ms, 2.2 ms]	998.596 µs (84.7%)
tracing	1.82 ms [1.805 ms, 1.836 ms]	641.164 µs (54.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.19 ms [1.178 ms, 1.202 ms]	-
iast	3.158 ms [3.119 ms, 3.198 ms]	1.969 ms (165.4%)
iast_FULL	5.833 ms [5.774 ms, 5.891 ms]	4.643 ms (390.2%)
iast_GLOBAL	3.688 ms [3.627 ms, 3.749 ms]	2.498 ms (209.9%)
profiling	2.133 ms [2.113 ms, 2.153 ms]	943.096 µs (79.3%)
tracing	1.785 ms [1.77 ms, 1.8 ms]	595.106 µs (50.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/test-merge-base-benchmark
git_commit_date	1767714789	1767884162
git_commit_sha	`3917d52`	`9da26aa`
release_version	1.59.0-SNAPSHOT~3917d52aab	1.57.0-SNAPSHOT~9da26aafd7

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1767886453	1767886453
ci_job_id	1336196542	1336196542
ci_pipeline_id	89677605	89677605
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-e3k5axlf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-e3k5axlf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (3.723 ms) : 3503, 3943
.   : milestone, 3723,
iast (2.216 ms) : 2151, 2280
.   : milestone, 2216,
iast_GLOBAL (2.267 ms) : 2202, 2332
.   : milestone, 2267,
profiling (2.071 ms) : 2018, 2123
.   : milestone, 2071,
tracing (2.051 ms) : 2000, 2101
.   : milestone, 2051,
section candidate
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (3.674 ms) : 3459, 3890
.   : milestone, 3674,
iast (2.214 ms) : 2149, 2278
.   : milestone, 2214,
iast_GLOBAL (2.263 ms) : 2198, 2328
.   : milestone, 2263,
profiling (2.072 ms) : 2019, 2125
.   : milestone, 2072,
tracing (2.046 ms) : 1995, 2097
.   : milestone, 2046,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.465 ms, 1.488 ms]	-
appsec	3.723 ms [3.503 ms, 3.943 ms]	2.246 ms (152.1%)
iast	2.216 ms [2.151 ms, 2.28 ms]	739.039 µs (50.0%)
iast_GLOBAL	2.267 ms [2.202 ms, 2.332 ms]	789.931 µs (53.5%)
profiling	2.071 ms [2.018 ms, 2.123 ms]	594.119 µs (40.2%)
tracing	2.051 ms [2.0 ms, 2.101 ms]	573.818 µs (38.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.47 ms [1.459 ms, 1.482 ms]	-
appsec	3.674 ms [3.459 ms, 3.89 ms]	2.204 ms (149.9%)
iast	2.214 ms [2.149 ms, 2.278 ms]	743.228 µs (50.5%)
iast_GLOBAL	2.263 ms [2.198 ms, 2.328 ms]	792.195 µs (53.9%)
profiling	2.072 ms [2.019 ms, 2.125 ms]	601.742 µs (40.9%)
tracing	2.046 ms [1.995 ms, 2.097 ms]	575.515 µs (39.1%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~9da26aafd7, baseline=1.59.0-SNAPSHOT~3917d52aab
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.6 s) : 15600000, 15600000
.   : milestone, 15600000,
appsec (14.404 s) : 14404000, 14404000
.   : milestone, 14404000,
iast (17.949 s) : 17949000, 17949000
.   : milestone, 17949000,
iast_GLOBAL (17.817 s) : 17817000, 17817000
.   : milestone, 17817000,
profiling (14.82 s) : 14820000, 14820000
.   : milestone, 14820000,
tracing (14.513 s) : 14513000, 14513000
.   : milestone, 14513000,
section candidate
no_agent (14.895 s) : 14895000, 14895000
.   : milestone, 14895000,
appsec (14.844 s) : 14844000, 14844000
.   : milestone, 14844000,
iast (18.496 s) : 18496000, 18496000
.   : milestone, 18496000,
iast_GLOBAL (18.176 s) : 18176000, 18176000
.   : milestone, 18176000,
profiling (15.054 s) : 15054000, 15054000
.   : milestone, 15054000,
tracing (14.428 s) : 14428000, 14428000
.   : milestone, 14428000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.6 s [15.6 s, 15.6 s]	-
appsec	14.404 s [14.404 s, 14.404 s]	-1.196 s (-7.7%)
iast	17.949 s [17.949 s, 17.949 s]	2.349 s (15.1%)
iast_GLOBAL	17.817 s [17.817 s, 17.817 s]	2.217 s (14.2%)
profiling	14.82 s [14.82 s, 14.82 s]	-780.0 ms (-5.0%)
tracing	14.513 s [14.513 s, 14.513 s]	-1.087 s (-7.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.895 s [14.895 s, 14.895 s]	-
appsec	14.844 s [14.844 s, 14.844 s]	-51.0 ms (-0.3%)
iast	18.496 s [18.496 s, 18.496 s]	3.601 s (24.2%)
iast_GLOBAL	18.176 s [18.176 s, 18.176 s]	3.281 s (22.0%)
profiling	15.054 s [15.054 s, 15.054 s]	159.0 ms (1.1%)
tracing	14.428 s [14.428 s, 14.428 s]	-467.0 ms (-3.1%)

sarahchen6 · 2025-12-19T21:22:46Z

Now testing https://github.com/DataDog/apm-sdks-benchmarks/pull/45

sarahchen6 added the tag: do not merge Do not merge changes label Dec 12, 2025

sarahchen6 temporarily deployed to ci-build December 12, 2025 22:48 — with GitHub Actions Inactive

sarahchen6 added 2 commits December 12, 2025 16:51

Add apm-sdks-benchmarks to CI

d9d4887

Add thresholds file

732aefd

sarahchen6 force-pushed the sarahchen6/test-merge-base-benchmark branch from c1cba5f to 732aefd Compare December 12, 2025 23:03

sarahchen6 added 2 commits December 12, 2025 17:10

Empty commit to run benchmarks again

0beddad

Empty commit to run benchmarks again

9610907

sarahchen6 had a problem deploying to ci-build December 12, 2025 23:20 — with GitHub Actions Error

sarahchen6 temporarily deployed to ci-build December 12, 2025 23:21 — with GitHub Actions Inactive

Empty commit to run benchmarks again

4fc38a5

sarahchen6 temporarily deployed to ci-build December 12, 2025 23:50 — with GitHub Actions Inactive

Empty commit to run benchmarks again

bb85a00

sarahchen6 temporarily deployed to ci-build December 12, 2025 23:54 — with GitHub Actions Inactive

Empty commit to run benchmarks again

e6573d0

sarahchen6 temporarily deployed to ci-build December 13, 2025 00:00 — with GitHub Actions Inactive

sarahchen6 added 2 commits January 6, 2026 12:20

Merge branch 'master' into sarahchen6/test-merge-base-benchmark

a4e4452

Run microbenchmarks

573ddbc

sarahchen6 temporarily deployed to ci-build January 6, 2026 17:21 — with GitHub Actions Inactive

Add infrastructure stage

d3ae881

sarahchen6 had a problem deploying to ci-build January 6, 2026 21:54 — with GitHub Actions Error

Add another stage

b28e0fa

sarahchen6 temporarily deployed to ci-build January 6, 2026 21:58 — with GitHub Actions Inactive

Empty commit to run benchmarks

177d428

sarahchen6 temporarily deployed to ci-build January 6, 2026 23:11 — with GitHub Actions Inactive

Empty commit to run benchmarks again

1fb5e0c

sarahchen6 temporarily deployed to ci-build January 6, 2026 23:42 — with GitHub Actions Inactive

Include insecure bank file

dbd6c94

sarahchen6 temporarily deployed to ci-build January 7, 2026 20:47 — with GitHub Actions Inactive

sarahchen6 added 2 commits January 7, 2026 15:56

Empty commit to run benchmarks without full suite

9a0a7d5

Empty commit to run benchmarks with updated file

22465b7

sarahchen6 had a problem deploying to ci-build January 7, 2026 21:20 — with GitHub Actions Error

sarahchen6 temporarily deployed to ci-build January 7, 2026 21:26 — with GitHub Actions Inactive

Empty commit to run benchmarks again

63e044f

sarahchen6 temporarily deployed to ci-build January 7, 2026 22:33 — with GitHub Actions Inactive

Empty commit to run benchmarks again

b279363

sarahchen6 temporarily deployed to ci-build January 7, 2026 23:05 — with GitHub Actions Inactive

Empty commit to run benchmarks again...

f501067

sarahchen6 temporarily deployed to ci-build January 7, 2026 23:37 — with GitHub Actions Inactive

Empty commit to run benchmarks again again

9da26aa

sarahchen6 temporarily deployed to ci-build January 8, 2026 14:56 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `apm-sdks-benchmarks` to CI #10206

Add `apm-sdks-benchmarks` to CI #10206

Uh oh!

sarahchen6 commented Dec 12, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Dec 13, 2025 •

edited

Loading

Uh oh!

sarahchen6 commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add apm-sdks-benchmarks to CI #10206

Are you sure you want to change the base?

Add apm-sdks-benchmarks to CI #10206

Uh oh!

Conversation

sarahchen6 commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

sarahchen6 commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add `apm-sdks-benchmarks` to CI #10206

Add `apm-sdks-benchmarks` to CI #10206

sarahchen6 commented Dec 12, 2025 •

edited

Loading

pr-commenter bot commented Dec 13, 2025 •

edited

Loading