Skip to content

Route GitLab CI Java dependency downloads through Depot - #866

Open
zihaoyu wants to merge 2 commits into
DataDog:mainfrom
zihaoyu:codex/depot-java-dependencies
Open

zihaoyu wants to merge 2 commits into
DataDog:mainfrom
zihaoyu:codex/depot-java-dependencies

Conversation

@zihaoyu

@zihaoyu zihaoyu commented Oct 15, 2026 •

Copy link
Copy Markdown

What does this PR do?:

Route Java dependency resolution through Depot in GitLab CI only, covering the main Gradle build, included build-logic build, and JFR validation tooling. External contributors retain the existing repository and launcher behavior when GITLAB_CI is unset or is not exactly true.

  • In GitLab CI, Gradle reads MAVEN_REPOSITORY_PROXY and GRADLE_PLUGIN_PROXY, with explicit mavenRepositoryProxy / gradlePluginProxy Gradle properties taking precedence. Configured proxies replace the explicit public repository fallbacks.
  • Configure Depot's Gradle Plugin Portal mirror alongside Java Depot. Plugin-only artifacts such as the Kotlin DSL 6.7.6 marker require that separate mirror.
  • In GitLab CI, the shared JBang launcher pins Java 25-compatible Jafar 0.26.2, resolves both io.btrace:jafar-shell and its matching io.btrace:jfr-shell-jafar backend through the configured repository, and ignores repositories declared by transitive dependencies. Supplying the backend prevents its separate embedded resolver from contacting public Maven repositories.
  • Use that launcher for prerequisite pre-warming and JFR conformance validation. Preserve metadata fallback, validation arguments, output modes, and failure propagation; use an argument array instead of shell-string evaluation.
  • Add nine launcher/conformance tests to the shell-unit-test CI job, assertions for four Gradle repository scopes, and documentation.

Contributor compatibility:

Shared Gradle and validation files are changed, but the strict Depot configuration and pinned Jafar/backend path are guarded by GITLAB_CI=true.

  • Local builds and GitHub Actions retain Maven Central and the Gradle Plugin Portal in their original order. The existing optional -PmavenRepositoryProxy=... remains supported with its original public fallbacks.
  • Outside GitLab CI, the new Gradle environment-variable handling and plugin-proxy property do not activate.
  • Outside GitLab CI, JFR validation still uses jbang --java 25 jfr-shell@btraceio, including its existing optional MAVEN_REPOSITORY_PROXY override. It does not force the CI version pin, backend dependency, or transitive-repository restriction.
  • Forks building locally or through GitHub Actions need no Depot access. A fork explicitly running GitLab CI inherits the GitLab-specific configuration.

Motivation:

The CI execution external network traffic dashboard, filtered to repo1.maven.org identified direct Maven Central traffic from java-profiler, including main. This follows the Dependency Management Slack discussion about migrating JVM downloads to Depot.

The existing CI configuration left public repository fallbacks enabled, some included-build declarations consumed only Gradle properties rather than the CI environment, and JBang/Jafar had additional catalog and backend resolution paths. This PR closes those gaps for current main; older branches are outside its scope.

Additional Notes:

  • No native profiler implementation, production runtime options, signing, publishing, or credential handling changes.
  • Depot can redirect cache misses upstream. The change routes initial resolution through Depot and removes configured public fallbacks in CI; it does not guarantee every upstream redirect disappears.
  • The native test suite is not green. Five cases failed in the latest local run, matching the earlier patched run. Only the Vtable failure has been reproduced on unchanged main; the other four are not claimed to be pre-existing.

How to test the change?:

Validated locally on macOS / JDK 25:

Check Result
Launcher/conformance Python tests PASS: 9 tests. Covers GitLab routing, non-GitLab catalog/default/custom-repository behavior, metadata fallback, argument/output preservation, validation failure, and missing recordings.
Contributor regression proof The three new non-GitLab compatibility tests fail against the preceding PR revision and pass after the GitLab guard.
Shell syntax and git diff --check PASS. Launcher, conformance script, and prerequisite installer syntax checked.
GitLab Depot configuration PASS: all four repository assertions during help: main plugins, project dependencies, included-build plugins, convention dependencies.
GitHub Actions / public configuration PASS: GITLAB_CI unset, GITHUB_ACTIONS=true, deliberately invalid proxy environment URLs. All four original public repository lists verified.
External explicit custom proxy PASS: GITLAB_CI=false, explicit Maven proxy property, deliberately invalid proxy environment URLs. All four lists retain the custom proxy followed by original public fallbacks.
Explicit property precedence PASS: valid proxy properties override deliberately invalid environment URLs; all four Depot assertions pass in the native and final formatting runs.
Final spotlessApply help through the repository wrapper PASS, 2s.
Real JBang/Jafar smoke checks from initial Depot validation Shell and backend resolved through Depot, an empty script ran, and metadata was read from a real JDK 25 recording. The CI launcher resolution path is retained.
Real conformance smoke check from initial validation Ran 425 commands and correctly rejected a small recording lacking required execution samples. This verifies execution and failure reporting, not a passing profiler recording.
Latest full testDebug FAILED in 6m39s: 165 passed, 5 failed. Console reports 235 total / 65 skipped; XML reports 236 total / 66 skipped, including an additional skipped entry. Pass/failure counts agree. Dependency resolution and Java/native compilation completed.
Unchanged-main comparison Targeted VtableReceiverFrameTest reproduces the same assertion on unchanged main at b302ee4169070dc9bf59f90959191787fe406f61. Other failures were not baseline-tested.

Latest native failures in :ddprof-test:testDebug:

  • NativeThreadTest [3]: no datadog.ExecutionSample for CPU profiling.
  • DynamicNativeThread [3]: no datadog.ExecutionSample with DWARF stacks.
  • ThreadEntryDetectionTest [3]: no datadog.ExecutionSample for CPU profiling.
  • VirtualThreadWallClockTest, cstack=vmx: no sample containing carrier-thread frames.
  • VtableReceiverFrameTest [5]: no expected vtable stub/receiver frames; also reproduced on unchanged main.

The local native run uses an uncommitted init script with -Djdk.attach.allowAttachSelf=true, -Djol.tryWithSudo=false, and -XX:+EnableDynamicAgentLoading to avoid JOL's local sudo-attach stall. Google Test was unavailable, so gtest tasks were skipped. A full GitLab pipeline has not been run as part of local validation.

Local evidence retained in the working checkout:

  • build/reports/claude/gradle-summary.md and gradle-summary.json
  • build/logs/20261015-160437-help.log — GitLab Depot configuration
  • build/logs/20261015-160619-help.log — GitHub Actions/public configuration
  • build/logs/20261015-160642-help.log — external custom proxy
  • build/logs/20261015-162158-spotlessApply.log — final formatting/help
  • build/logs/20261015-160745-testDebug.log — latest native run
  • build/logs/20261015-133604-testDebug.log — earlier patched native run
  • build/logs/baseline-20261015-134200-testDebug.log — targeted unchanged-main baseline

To repeat configuration assertions, use ./.claude/commands/build-and-summarize help -I .gitlab/scripts/tests/depot-repositories.init.gradle. Select DEPOT_EXPECTED_MODE=depot, public, or custom-proxy; provide expected proxy URLs independently of the inputs being tested. The README documents the modes. Run testDebug through the same wrapper on supported CI environments to investigate the remaining native failures.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a security review.
  • This PR does not change signing, publishing, or credential-handling logic.
  • JIRA: not linked.

@zihaoyu zihaoyu changed the title Route Gradle and JFR validation dependencies through Depot Route GitLab CI Java dependency downloads through Depot Oct 15, 2026
@zihaoyu
zihaoyu marked this pull request as ready for review October 15, 2026 20:31
@zihaoyu
zihaoyu requested a review from a team as a code owner October 15, 2026 20:31
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T20:36:58.888569Z 031db5a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant