Skip to content

chore: Pin system-tests for release/v1.58.x branch #10292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
mcculls merged 1 commit into from
Jan 6, 2026
Merged

chore: Pin system-tests for release/v1.58.x branch #10292

mcculls merged 1 commit into from
Jan 6, 2026

Conversation

@mcculls
Copy link
Contributor

@mcculls mcculls commented Jan 6, 2026

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mcculls mcculls requested a review from a team as a code owner January 6, 2026 12:03
@mcculls mcculls requested review from AlexeyKuznetsov-DD and removed request for a team January 6, 2026 12:03
@mcculls mcculls added comp: testing Testing tag: no release notes Changes to exclude from release notes type: refactoring labels Jan 6, 2026
@mcculls mcculls requested a review from PerfectSlayer January 6, 2026 12:05
@PerfectSlayer
Copy link
Contributor

PerfectSlayer commented Jan 6, 2026

Holding on the PR until we found out why the automation did not run.

@pr-commenter
Copy link

pr-commenter bot commented Jan 6, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/pin-system-tests-v1.58.x
git_commit_date 1767637205 1767700863
git_commit_sha bd3f6f5 7e59722
release_version 1.58.0~bd3f6f5c89 1.59.0-SNAPSHOT~7e59722f54
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1767702707 1767702707
ci_job_id 1329276989 1329276989
ci_pipeline_id 89174481 89174481
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-hocwd674 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-hocwd674 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 7 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1084869
Total [baseline] (10.878 s) : 0, 10877946
Agent [candidate] (1.084 s) : 0, 1083691
Total [candidate] (10.76 s) : 0, 10760377
section appsec
Agent [baseline] (1.266 s) : 0, 1266266
Total [baseline] (10.917 s) : 0, 10916637
Agent [candidate] (1.269 s) : 0, 1269273
Total [candidate] (10.995 s) : 0, 10994837
section iast
Agent [baseline] (1.227 s) : 0, 1226945
Total [baseline] (11.202 s) : 0, 11201826
Agent [candidate] (1.233 s) : 0, 1232770
Total [candidate] (11.177 s) : 0, 11177434
section profiling
Agent [baseline] (1.213 s) : 0, 1212586
Total [baseline] (11.09 s) : 0, 11090146
Agent [candidate] (1.204 s) : 0, 1203558
Total [candidate] (10.907 s) : 0, 10907058
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent appsec 1.266 s 181.397 ms (16.7%)
Agent iast 1.227 s 142.077 ms (13.1%)
Agent profiling 1.213 s 127.717 ms (11.8%)
Total tracing 10.878 s -
Total appsec 10.917 s 38.692 ms (0.4%)
Total iast 11.202 s 323.881 ms (3.0%)
Total profiling 11.09 s 212.2 ms (2.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.084 s -
Agent appsec 1.269 s 185.582 ms (17.1%)
Agent iast 1.233 s 149.079 ms (13.8%)
Agent profiling 1.204 s 119.868 ms (11.1%)
Total tracing 10.76 s -
Total appsec 10.995 s 234.46 ms (2.2%)
Total iast 11.177 s 417.057 ms (3.9%)
Total profiling 10.907 s 146.681 ms (1.4%) 
gantt
    title petclinic - break down per module: candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (650.994 ms) : 0, 650994
BytebuddyAgent [candidate] (650.711 ms) : 0, 650711
GlobalTracer [baseline] (282.787 ms) : 0, 282787
GlobalTracer [candidate] (282.604 ms) : 0, 282604
AppSec [baseline] (32.396 ms) : 0, 32396
AppSec [candidate] (32.433 ms) : 0, 32433
Debugger [baseline] (68.462 ms) : 0, 68462
Debugger [candidate] (68.03 ms) : 0, 68030
Remote Config [baseline] (630.225 µs) : 0, 630
Remote Config [candidate] (625.151 µs) : 0, 625
Telemetry [baseline] (9.08 ms) : 0, 9080
Telemetry [candidate] (8.956 ms) : 0, 8956
Flare Poller [baseline] (3.718 ms) : 0, 3718
Flare Poller [candidate] (3.691 ms) : 0, 3691
section appsec
crashtracking [baseline] (1.182 ms) : 0, 1182
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (691.403 ms) : 0, 691403
BytebuddyAgent [candidate] (693.863 ms) : 0, 693863
GlobalTracer [baseline] (258.341 ms) : 0, 258341
GlobalTracer [candidate] (257.995 ms) : 0, 257995
AppSec [baseline] (172.036 ms) : 0, 172036
AppSec [candidate] (174.782 ms) : 0, 174782
Debugger [baseline] (69.251 ms) : 0, 69251
Debugger [candidate] (67.281 ms) : 0, 67281
Remote Config [baseline] (797.545 µs) : 0, 798
Remote Config [candidate] (771.589 µs) : 0, 772
Telemetry [baseline] (9.547 ms) : 0, 9547
Telemetry [candidate] (9.382 ms) : 0, 9382
Flare Poller [baseline] (3.796 ms) : 0, 3796
Flare Poller [candidate] (3.769 ms) : 0, 3769
IAST [baseline] (24.428 ms) : 0, 24428
IAST [candidate] (24.641 ms) : 0, 24641
section iast
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (793.782 ms) : 0, 793782
BytebuddyAgent [candidate] (797.73 ms) : 0, 797730
GlobalTracer [baseline] (256.467 ms) : 0, 256467
GlobalTracer [candidate] (257.64 ms) : 0, 257640
AppSec [baseline] (35.015 ms) : 0, 35015
AppSec [candidate] (33.6 ms) : 0, 33600
Debugger [baseline] (65.403 ms) : 0, 65403
Debugger [candidate] (67.008 ms) : 0, 67008
Remote Config [baseline] (583.818 µs) : 0, 584
Remote Config [candidate] (587.66 µs) : 0, 588
Telemetry [baseline] (8.496 ms) : 0, 8496
Telemetry [candidate] (8.6 ms) : 0, 8600
Flare Poller [baseline] (3.624 ms) : 0, 3624
Flare Poller [candidate] (3.614 ms) : 0, 3614
IAST [baseline] (26.895 ms) : 0, 26895
IAST [candidate] (27.215 ms) : 0, 27215
section profiling
crashtracking [baseline] (1.209 ms) : 0, 1209
crashtracking [candidate] (1.213 ms) : 0, 1213
BytebuddyAgent [baseline] (705.413 ms) : 0, 705413
BytebuddyAgent [candidate] (702.342 ms) : 0, 702342
GlobalTracer [baseline] (222.512 ms) : 0, 222512
GlobalTracer [candidate] (220.322 ms) : 0, 220322
AppSec [baseline] (32.855 ms) : 0, 32855
AppSec [candidate] (32.027 ms) : 0, 32027
Debugger [baseline] (68.782 ms) : 0, 68782
Debugger [candidate] (68.275 ms) : 0, 68275
Remote Config [baseline] (668.018 µs) : 0, 668
Remote Config [candidate] (618.883 µs) : 0, 619
Telemetry [baseline] (8.852 ms) : 0, 8852
Telemetry [candidate] (8.763 ms) : 0, 8763
Flare Poller [baseline] (3.747 ms) : 0, 3747
Flare Poller [candidate] (3.646 ms) : 0, 3646
ProfilingAgent [baseline] (98.704 ms) : 0, 98704
ProfilingAgent [candidate] (96.601 ms) : 0, 96601
Profiling [baseline] (99.303 ms) : 0, 99303
Profiling [candidate] (97.174 ms) : 0, 97174
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.094 s) : 0, 1094103
Total [baseline] (8.838 s) : 0, 8837697
Agent [candidate] (1.085 s) : 0, 1085287
Total [candidate] (8.76 s) : 0, 8759878
section iast
Agent [baseline] (1.222 s) : 0, 1221822
Total [baseline] (9.295 s) : 0, 9295244
Agent [candidate] (1.232 s) : 0, 1232284
Total [candidate] (9.362 s) : 0, 9362124
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.094 s -
Agent iast 1.222 s 127.719 ms (11.7%)
Total tracing 8.838 s -
Total iast 9.295 s 457.546 ms (5.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent iast 1.232 s 146.996 ms (13.5%)
Total tracing 8.76 s -
Total iast 9.362 s 602.246 ms (6.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.209 ms) : 0, 1209
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (656.685 ms) : 0, 656685
BytebuddyAgent [candidate] (653.371 ms) : 0, 653371
GlobalTracer [baseline] (285.194 ms) : 0, 285194
GlobalTracer [candidate] (282.447 ms) : 0, 282447
AppSec [baseline] (33.184 ms) : 0, 33184
AppSec [candidate] (32.449 ms) : 0, 32449
Debugger [baseline] (68.467 ms) : 0, 68467
Debugger [candidate] (66.958 ms) : 0, 66958
Remote Config [baseline] (630.086 µs) : 0, 630
Remote Config [candidate] (617.947 µs) : 0, 618
Telemetry [baseline] (9.082 ms) : 0, 9082
Telemetry [candidate] (8.983 ms) : 0, 8983
Flare Poller [baseline] (3.787 ms) : 0, 3787
Flare Poller [candidate] (3.704 ms) : 0, 3704
section iast
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (789.948 ms) : 0, 789948
BytebuddyAgent [candidate] (797.565 ms) : 0, 797565
GlobalTracer [baseline] (255.768 ms) : 0, 255768
GlobalTracer [candidate] (257.646 ms) : 0, 257646
AppSec [baseline] (33.726 ms) : 0, 33726
AppSec [candidate] (34.473 ms) : 0, 34473
Debugger [baseline] (66.028 ms) : 0, 66028
Debugger [candidate] (65.692 ms) : 0, 65692
Remote Config [baseline] (577.56 µs) : 0, 578
Remote Config [candidate] (597.889 µs) : 0, 598
Telemetry [baseline] (8.505 ms) : 0, 8505
Telemetry [candidate] (8.612 ms) : 0, 8612
Flare Poller [baseline] (3.559 ms) : 0, 3559
Flare Poller [candidate] (3.621 ms) : 0, 3621
IAST [baseline] (27.216 ms) : 0, 27216
IAST [candidate] (27.31 ms) : 0, 27310
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/pin-system-tests-v1.58.x
git_commit_date 1767637205 1767700863
git_commit_sha bd3f6f5 7e59722
release_version 1.58.0~bd3f6f5c89 1.59.0-SNAPSHOT~7e59722f54
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1767703190 1767703190
ci_job_id 1329276991 1329276991
ci_pipeline_id 89174481 89174481
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-ij2nur7l 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-ij2nur7l 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 1 performance regressions! Performance is the same for 17 metrics, 17 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load better
[-438.004µs; -137.106µs] or [-8.179%; -2.560%]		 same
[-866.683µs; +46.749µs] or [-6.817%; +0.368%]		 unstable
[-47.739op/s; +119.177op/s] or [-6.183%; +15.435%]		 5.068ms 12.304ms 807.844op/s 5.355ms 12.714ms 772.125op/s
scenario:load:petclinic:no_agent:high_load worse
[+1.312ms; +2.603ms] or [+7.853%; +15.580%]		 unstable
[+1.249ms; +4.386ms] or [+4.392%; +15.416%]		 unstable
[-45.403op/s; +5.338op/s] or [-16.816%; +1.977%]		 18.665ms 31.267ms 249.968op/s 16.708ms 28.449ms 270.000op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.186 ms) : 1175, 1197
.   : milestone, 1186,
iast (3.163 ms) : 3115, 3211
.   : milestone, 3163,
iast_FULL (5.992 ms) : 5931, 6052
.   : milestone, 5992,
iast_GLOBAL (3.631 ms) : 3570, 3693
.   : milestone, 3631,
profiling (2.019 ms) : 2002, 2036
.   : milestone, 2019,
tracing (1.751 ms) : 1736, 1766
.   : milestone, 1751,
section candidate
no_agent (1.182 ms) : 1171, 1193
.   : milestone, 1182,
iast (3.101 ms) : 3062, 3141
.   : milestone, 3101,
iast_FULL (5.722 ms) : 5664, 5780
.   : milestone, 5722,
iast_GLOBAL (3.53 ms) : 3481, 3580
.   : milestone, 3530,
profiling (2.05 ms) : 2032, 2068
.   : milestone, 2050,
tracing (1.756 ms) : 1742, 1771
.   : milestone, 1756,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.186 ms [1.175 ms, 1.197 ms] -
iast 3.163 ms [3.115 ms, 3.211 ms] 1.977 ms (166.6%)
iast_FULL 5.992 ms [5.931 ms, 6.052 ms] 4.805 ms (405.1%)
iast_GLOBAL 3.631 ms [3.57 ms, 3.693 ms] 2.445 ms (206.1%)
profiling 2.019 ms [2.002 ms, 2.036 ms] 833.13 µs (70.2%)
tracing 1.751 ms [1.736 ms, 1.766 ms] 564.764 µs (47.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.182 ms [1.171 ms, 1.193 ms] -
iast 3.101 ms [3.062 ms, 3.141 ms] 1.919 ms (162.3%)
iast_FULL 5.722 ms [5.664 ms, 5.78 ms] 4.54 ms (384.0%)
iast_GLOBAL 3.53 ms [3.481 ms, 3.58 ms] 2.348 ms (198.6%)
profiling 2.05 ms [2.032 ms, 2.068 ms] 867.235 µs (73.3%)
tracing 1.756 ms [1.742 ms, 1.771 ms] 573.875 µs (48.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.28 ms) : 17103, 17456
.   : milestone, 17280,
appsec (19.401 ms) : 19204, 19598
.   : milestone, 19401,
code_origins (17.739 ms) : 17562, 17916
.   : milestone, 17739,
iast (18.512 ms) : 18324, 18701
.   : milestone, 18512,
profiling (18.444 ms) : 18262, 18626
.   : milestone, 18444,
tracing (17.691 ms) : 17516, 17866
.   : milestone, 17691,
section candidate
no_agent (19.274 ms) : 19073, 19475
.   : milestone, 19274,
appsec (19.278 ms) : 19078, 19477
.   : milestone, 19278,
code_origins (18.073 ms) : 17887, 18259
.   : milestone, 18073,
iast (18.531 ms) : 18341, 18721
.   : milestone, 18531,
profiling (18.635 ms) : 18445, 18824
.   : milestone, 18635,
tracing (17.58 ms) : 17407, 17753
.   : milestone, 17580,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.28 ms [17.103 ms, 17.456 ms] -
appsec 19.401 ms [19.204 ms, 19.598 ms] 2.121 ms (12.3%)
code_origins 17.739 ms [17.562 ms, 17.916 ms] 459.085 µs (2.7%)
iast 18.512 ms [18.324 ms, 18.701 ms] 1.233 ms (7.1%)
profiling 18.444 ms [18.262 ms, 18.626 ms] 1.164 ms (6.7%)
tracing 17.691 ms [17.516 ms, 17.866 ms] 411.403 µs (2.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.274 ms [19.073 ms, 19.475 ms] -
appsec 19.278 ms [19.078 ms, 19.477 ms] 3.628 µs (0.0%)
code_origins 18.073 ms [17.887 ms, 18.259 ms] -1.201 ms (-6.2%)
iast 18.531 ms [18.341 ms, 18.721 ms] -743.003 µs (-3.9%)
profiling 18.635 ms [18.445 ms, 18.824 ms] -639.642 µs (-3.3%)
tracing 17.58 ms [17.407 ms, 17.753 ms] -1.694 ms (-8.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/pin-system-tests-v1.58.x
git_commit_date 1767637205 1767700863
git_commit_sha bd3f6f5 7e59722
release_version 1.58.0~bd3f6f5c89 1.59.0-SNAPSHOT~7e59722f54
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1767702970 1767702970
ci_job_id 1329276992 1329276992
ci_pipeline_id 89174481 89174481
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-zbsh8rmi 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-zbsh8rmi 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.368ms; -1.029ms] or [-37.250%; -28.018%]		 2.474ms 3.673ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (3.673 ms) : 3456, 3890
.   : milestone, 3673,
iast (2.219 ms) : 2154, 2284
.   : milestone, 2219,
iast_GLOBAL (2.258 ms) : 2193, 2324
.   : milestone, 2258,
profiling (2.092 ms) : 2038, 2147
.   : milestone, 2092,
tracing (2.044 ms) : 1993, 2096
.   : milestone, 2044,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.474 ms) : 2422, 2527
.   : milestone, 2474,
iast (2.219 ms) : 2154, 2285
.   : milestone, 2219,
iast_GLOBAL (2.257 ms) : 2192, 2322
.   : milestone, 2257,
profiling (2.068 ms) : 2016, 2121
.   : milestone, 2068,
tracing (2.054 ms) : 2003, 2105
.   : milestone, 2054,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 3.673 ms [3.456 ms, 3.89 ms] 2.193 ms (148.1%)
iast 2.219 ms [2.154 ms, 2.284 ms] 738.521 µs (49.9%)
iast_GLOBAL 2.258 ms [2.193 ms, 2.324 ms] 778.033 µs (52.6%)
profiling 2.092 ms [2.038 ms, 2.147 ms] 611.91 µs (41.3%)
tracing 2.044 ms [1.993 ms, 2.096 ms] 564.157 µs (38.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.461 ms, 1.484 ms] -
appsec 2.474 ms [2.422 ms, 2.527 ms] 1.002 ms (68.1%)
iast 2.219 ms [2.154 ms, 2.285 ms] 747.314 µs (50.8%)
iast_GLOBAL 2.257 ms [2.192 ms, 2.322 ms] 785.057 µs (53.3%)
profiling 2.068 ms [2.016 ms, 2.121 ms] 596.297 µs (40.5%)
tracing 2.054 ms [2.003 ms, 2.105 ms] 581.391 µs (39.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~7e59722f54, baseline=1.58.0~bd3f6f5c89
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.791 s) : 14791000, 14791000
.   : milestone, 14791000,
appsec (14.573 s) : 14573000, 14573000
.   : milestone, 14573000,
iast (18.271 s) : 18271000, 18271000
.   : milestone, 18271000,
iast_GLOBAL (18.629 s) : 18629000, 18629000
.   : milestone, 18629000,
profiling (14.775 s) : 14775000, 14775000
.   : milestone, 14775000,
tracing (14.431 s) : 14431000, 14431000
.   : milestone, 14431000,
section candidate
no_agent (14.964 s) : 14964000, 14964000
.   : milestone, 14964000,
appsec (14.86 s) : 14860000, 14860000
.   : milestone, 14860000,
iast (18.391 s) : 18391000, 18391000
.   : milestone, 18391000,
iast_GLOBAL (18.03 s) : 18030000, 18030000
.   : milestone, 18030000,
profiling (14.806 s) : 14806000, 14806000
.   : milestone, 14806000,
tracing (14.716 s) : 14716000, 14716000
.   : milestone, 14716000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.791 s [14.791 s, 14.791 s] -
appsec 14.573 s [14.573 s, 14.573 s] -218.0 ms (-1.5%)
iast 18.271 s [18.271 s, 18.271 s] 3.48 s (23.5%)
iast_GLOBAL 18.629 s [18.629 s, 18.629 s] 3.838 s (25.9%)
profiling 14.775 s [14.775 s, 14.775 s] -16.0 ms (-0.1%)
tracing 14.431 s [14.431 s, 14.431 s] -360.0 ms (-2.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.964 s [14.964 s, 14.964 s] -
appsec 14.86 s [14.86 s, 14.86 s] -104.0 ms (-0.7%)
iast 18.391 s [18.391 s, 18.391 s] 3.427 s (22.9%)
iast_GLOBAL 18.03 s [18.03 s, 18.03 s] 3.066 s (20.5%)
profiling 14.806 s [14.806 s, 14.806 s] -158.0 ms (-1.1%)
tracing 14.716 s [14.716 s, 14.716 s] -248.0 ms (-1.7%)

@mcculls
Copy link
Contributor Author

mcculls commented Jan 6, 2026 •
edited
Loading

Note: currently the pinned system tests are failing due to an issue downloading maven-metadata.xml from Maven central - this then leads to an error resolving a Maven build plugin: https://github.com/DataDog/dd-trace-java/actions/runs/20747716337/job/59582228636

The Maven central issue appears to be resolved

@mcculls
Copy link
Contributor Author

mcculls commented Jan 6, 2026

Holding on the PR until we found out why the automation did not run.

@PerfectSlayer OK to merge this now?

@PerfectSlayer
Copy link
Contributor

PerfectSlayer commented Jan 6, 2026

It should be okay as @sarahchen6 has a way to keep investigating so we can merge this PR (I wanted to throw away any chance of investigation if we were able to recover).
By the way, the automation can also be triggered manually - in case where we want to update the pin. So the manual trigger can also be used in case failures 😇

@mcculls mcculls merged commit 0ba80b6 into release/v1.58.x Jan 6, 2026
593 of 630 checks passed
@mcculls mcculls deleted the mcculls/pin-system-tests-v1.58.x branch January 6, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@amarziali amarziali amarziali approved these changes

@sarahchen6 sarahchen6 sarahchen6 approved these changes

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD Awaiting requested review from AlexeyKuznetsov-DD AlexeyKuznetsov-DD is a code owner automatically assigned from DataDog/apm-lang-platform-java

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@mcculls @PerfectSlayer @amarziali @sarahchen6