Repository navigation
Conversation
…registry
With fjmethodid=lazy (HotSpot, cstack=vm), no jmethodIDs are preloaded at
ClassPrepare. Instead every class is recorded in KlassRegistry, a Klass* to
weak-mirror table. walkVM keeps storing raw Method* frames for classes
without jmethodIDs; at dump time resolve() reads only the holder Klass* and
idnum from the Method*, pins the registered mirror (null if the class was
unloaded), checks that the Method* is among the pinned class's current
methods, and only then allocates jmethodIDs for that one class.
Falls back to the partial mode when the Klass*/methods vmStructs are
missing. Adds method_resolve_lazy_{unloaded,not_current,class_loaded}
counters, gtests for the registry, containsMethod, argument parsing and the
load-state selection, and a Java test covering custom-loader resolution and
frames of an unloaded class.
Klass* address reuse after class unloading is not handled yet; see
doc/LazyJmethodIdResolution.md (investigation notes, to be removed before
landing).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ass* addresses A class unloaded and replaced by a new class at the same Klass* (and Method*) addresses made frames sampled from the old class resolve as methods of the new one, and made both produce identical raw frames. KlassRegistry now keeps an epoch, bumped when a known Klass* gets a different mirror and when a rehash purges dead entries; each entry records the epoch it was registered at. walkVM stores the current epoch in every raw frame as a 26-bit tag (10 spare bci bits plus the top 16 bits of the Method*), and pin() refuses frames older than the entry. A Method* that uses the tag's address bits is stored as not walkable. If registering a class fails, its jmethodIDs are preloaded so it never yields raw frames. resolve() now takes the frame's bci to recover the tag. Adds method_resolve_lazy_stale, klass_registry_add_failed and walkvm_raw_frame_untaggable counters, an end-to-end gtest for the Klass*-reuse cases, epoch tests for the registry and tag round-trip tests for FrameType. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
KlassTraceId::calibrate() makes the JVM create six array klasses of java.lang.Void back to back and accepts the single 8-byte-aligned offset within sizeof(Klass) at which all of them hold distinct JFR trace IDs (raw >> 16) no further apart than consecutive creation allows; anything else, including an ambiguous match, leaves the offset unknown. Runs when fjmethodid=lazy is selected and publishes the result in the klass_trace_id_offset gauge. Not used by the sampler yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When KlassTraceId found Klass::_trace_id, walkVM now stores the low 26 bits of the holder's trace ID (read with SafeAccess via the new VMMethod::holderOrNull()) as the raw frame's tag instead of the registry epoch, and resolveViaRegistry() compares it with the pinned class's trace ID. The ID is unique per load of a class, so frames of an earlier class at a reused Klass* are refused without the epoch, and unrelated registry bumps no longer change a stack's frames. Frames whose holder trace ID is unreadable are stored as not walkable. The tag source is chosen once per process: calibration now runs only on the first lazy start, and the klass_trace_id_offset gauge is republished on every start since counters are reset then. The end-to-end lazy-resolve gtest runs every case under both tag sources. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st JFR in CI The top bit of a raw frame's tag now says whether the rest is a trace ID or a registry epoch (25 bits each), and resolve() goes by the frame's own kind rather than the current tag source. In debug builds, resolveViaRegistry() logs the trace ID it read for each class it allocates jmethodIDs for. LazyTraceIdJfrValidationTest runs a child JVM (new ExternalLauncher mode lazy-trace-id) with the profiler in lazy mode and JFR's own sampler, and requires JFR's RecordedClass.getId() for the probe class to equal that ID. The spin class generator moves to a shared SpinClasses test helper. The profiler does not call into JFR at runtime: JVM.getClassId() reaches JFR state that only exists once its recorder has been created, and the profiler itself initializes jdk.jfr.internal.JVM (tsc.cpp), so that class being initialized is no sign the recorder exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for fjmethodid=lazy: - VMKlass::containsMethod() reads the methods array only through SafeAccess. Pinning the class does not keep the array alive: a redefinition swaps in a new one at a safepoint and frees the old one at a later safepoint, neither of which waits for the native dump thread. - Redefinition: the RedefineClasses/RetransformClasses hooks mark the class in KlassRegistry before calling the original function and preload its jmethodIDs afterwards. pin() refuses raw frames of a marked class, and resolveViaRegistry() checks the mark again after its metadata reads, since a redefinition can free a Method* and reuse its address within the same class without changing its mirror, trace ID or epoch. New counter method_resolve_lazy_redefined; pin() now reports a PinStatus. - Every registration bumps the registry epoch. Previously only replacements and purges did, so a dead class's Method* reused by a class loaded at a different Klass* could match an epoch-tagged frame. - klass_trace_id_beyond_tag_range counts registrations of classes whose trace ID no longer fits in the 25-bit tag, the first point at which two classes' trace-ID tags can collide. - klassRegistry_ut includes vmStructs.inline.h so it links in release. The doc gets a "Guarantees and limits" section stating the exact condition under which a tag collision can still misattribute a frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion samples LazyTraceIdJfrValidationTest now records only jdk.ClassDefine in the child JVM (settings written by the test) and compares the probe class's RecordedClass.getId() from that event with the trace ID the profiler logged. JFR emits the event once per class on every JFR-enabled release, whereas JFR's pre-25 sampler recorded almost no jdk.ExecutionSample events for the spinning probe, even without the profiler running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lookup now keeps a per-dump map from (untagged Method*, raw-frame tag) to the resolved jmethodID, or nullptr for a refused frame, so a raw method that recurs across traces is resolved once per dump instead of once per occurrence. Each resolution sets up crash protection, takes the KlassRegistry mutex and pins the class through JNI. New counter raw_frame_resolve_cache_hits. Tests cover repeated occurrences with different bci and frame type, a fresh cache per dump, and frames from two loads of a class at the same addresses staying apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RedefineClassesHook and RetransformClassesHook now forward a call with a positive count and a null array straight to the original function, which returns JVMTI_ERROR_NULL_POINTER; before, the hooks' own loops dereferenced the array first and crashed, in every jmethodID mode. In fjmethodid=lazy, the per-class work before and after the call only touches elements that JVMTI's GetClassStatus accepts as instance classes (HotspotSupport::isInstanceClass, also used by registerClass), so a non-class object or an array class is never looked up as a Klass* or preloaded. redefineHooks_ut calls both hooks with stubbed originals in lazy and full-preload mode: null arrays, non-class, array-class and null elements. Without the fix its first case crashes with SIGSEGV. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LazyRedefinitionStressTest loads a small agent by self-attach to get Instrumentation, then redefines a generated class with its own bytes about every 2 ms while another thread spins in it, fjmethodid=lazy samples it and the profiler dumps every 100 ms. Raw frames of the class are collected for 300 ms before the first redefinition, so the first dumps afterwards have stale raw frames of a marked class to refuse. The test requires the JVM to survive, every redefinition to be marked, a stale raw frame to be refused, and frames sampled after the redefinitions to resolve by name. New counter klass_registry_redefinitions_marked counts the marks the redefinition hooks set. SpinClasses can hand out a class's bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raw frames of fjmethodid=lazy are now always tagged with the holder's JFR trace ID. When the trace-ID offset can't be calibrated (8u before 8u262, builds without JFR), lazy mode falls back to partial mode instead of tagging frames with the registry epoch. The 25-bit epoch wrapped after 2^24 registrations, after which every frame of a long-lived class was refused, and bumping it on every registration split identical stacks into separate traces. KlassRegistry no longer keeps an epoch. - Only lazy mode tags frames. Partial-mode frames are plain Method*s again, and resolve() picks the lazy or partial path from the frame's marker bit rather than the current mode, so frames that outlive a restart in the other mode still resolve the way they were sampled. - Redefinition: the hooks call KlassRegistry::beginRedefinition() before and endRedefinition() after the original function. A class is refused while a call runs and for good once one succeeds; a call JVMTI rejects leaves it as before and preloads nothing. Redefinitions are tracked in every mode once the registry has entries. - Mode selection uses the stack walker mode the profiler resolved, not the cstack argument: cstack=default resolves to dwarf where the VM walker is unavailable, and then all jmethodIDs must be preloaded. This also applied to fjmethodid=false. - VMKlass::containsMethod() scans at most 256 methods after the idnum fast path. - readMethodHolder() uses VMMethod::holderOrNull(). - The rehash() comment now matches what the code does. The prototype design notes (doc/LazyJmethodIdResolution.md) are removed; their content moves to the PR description. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The parse runs in the thread that reads the child JVM's output, so an exception there would stop that thread instead of failing the test. A value that does not parse now leaves the logged ID unset, which the existing assertion reports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Working notes for the branch, to be removed before the PR lands. Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HotspotSupport::resolve() now reads a raw Method* frame's class name, method name, signature, Thread.run status and line number table straight from HotSpot metadata (readMethod(), under crash protection) and returns them in a ResolvedMethod. In fjmethodid=lazy mode this runs while the class is pinned through KlassRegistry, after the trace-ID and current-method checks, so the dump no longer allocates jmethodIDs for sampled classes. Partial mode reads the same metadata instead of looking the method up through JNI FindClass/GetMethodID, which also covers <clinit> without a special case. - vmStructs: ConstMethod::_flags and _code_size, Klass::_super, the version-dependent has-line-number-table constant, and a decoder for the compressed line number table (VMLineNumberStream). - Lookup: raw frames get MethodMap rows keyed by the untagged Method* and tag bits (RAW_METHOD_MARK); refused frames never touch a row, and rows are refilled at their first use in every chunk. The JVMTI and VM-metadata paths share fillJavaMethodNames(); line tables decoded from VM metadata are malloc'd and freed with free(). - Counter method_resolve_lazy_class_loaded becomes method_resolve_lazy_resolved. - Tests: the resolve() crash-protection gtests and the line table, Thread.run and bci-bound cases; the lazy-resolution gtests now run on a class without jmethodIDs and check none are allocated; LazyLineNumberConsistencyTest compares decoded line numbers with JVMTI's; SpinClasses' generated method has a line number table. Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Not for merging. With DDPROF_CAPTURE_UNLOADING=1 in fjmethodid=lazy mode, the sampler records raw frames of classes outside the boot, platform and application loaders in a lock-free pending set; GarbageCollectionStart copies their names and line tables into a cache under crash protection; the dump uses a captured name when resolve() refuses a frame. Adds unload_capture_* counters, the benchmark harnesses used to measure it (utils/phase2-proto) and a hand-over note with the results so far. Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Not for merging. Adds a background drain to UnloadCapture: with
DDPROF_CAPTURE_UNLOADING=2 (drain + GC hook) or 3 (drain only), a daemon
thread resolves pending (Method*, tag) pairs every
DDPROF_CAPTURE_INTERVAL_MS (default 100) through the class registry,
via the new HotspotSupport::capturePinned(); resolveViaRegistry() gains
a flag to leave the method_resolve_lazy_* counters untouched for it.
Pending slots get a state word (free, pending, draining, done): drainers
claim pairs by CAS, captured pairs stay as DONE until endDump() so the
sampler does not add them again, and a sampler with no free slot evicts
a DONE one. The set grows to 16,384 slots.
Adds counters method_resolve_lazy_unreadable (a lazy Method* that could
not be read at dump time) and unload_capture_{bg_*,hook_drained,
pending_evicted}; Bench3 reports failed and lost percentages from them.
Adds driver scripts and Churn/RenWrap harnesses, and records the Linux
arm64 results and next steps in the hand-over note.
Environment: Datadog workspace
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
CI Test ResultsRun: #39256326416 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled Failed Jobs
Summary: Total: 32 | Passed: 0 | Failed: 32 Updated: 2026-10-19 13:06:20 UTC |
Contributor
|
🔄 Datadog auto-retried 1 job - 1 passed on retry 🔗 Commit SHA: 6a6000d | Docs | View more details | Give us feedback! |
Contributor
Reliability & Chaos Results✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/145637719 |
2 of 3 tasks
rkennke
force-pushed
the
investigate/lazy-jmethodid-vm-resolution
branch
2 times, most recently
from
October 21, 2026 09:58
922ace6 to
a154d17
Compare
Base automatically changed from
investigate/lazy-jmethodid-vm-resolution
to
main
October 21, 2026 15:56
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prototype, not for merging. Stacked on #868. It may never be followed through; this PR keeps the work and the measurements findable.
What does this PR do?:
Phase 2 of
doc/LazyJmethodIdCombinedPlan.md: keeps names forfjmethodid=lazyraw frames whose class is unloaded between the sample and the dump. Without it (#868 alone, and mainlinefjmethodid=true) those frames serialize as unknown.Opt-in through
DDPROF_CAPTURE_UNLOADINGin lazy mode, so one build measures every variant:1: GC hook.GarbageCollectionStartcopies the names and line tables of pending frames while the classes are still loaded (no JNI, no JVMTI, crash-protected).2: background drain + GC hook.3: background drain only. A daemon thread resolves pending frames everyDDPROF_CAPTURE_INTERVAL_MS(default 100) through the class registry, the same path the dump uses, outside any pause.Mechanics, in
hotspot/unloadCapture.{h,cpp}:Method*, tag) pairs (16,384 slots). Drainers claim entries by CAS, so the hook and the drain never take the same one. Captured pairs stay marked "done" until the next dump, so the sampler doesn't re-add them. A sampler that finds no free slot evicts a "done" slot.resolve()refuses a frame, the dump uses the captured name.unload_capture_*.Full results and next steps:
doc/LazyJmethodIdPhase2Handover.md. Harnesses:utils/phase2-proto/.Motivation:
#868 loses names of classes that unload before the dump, as mainline does. The question was whether recovering them is possible without increasing GC pause times.
Results (Linux arm64, 16 cores, JDK 21, release build):
-Xmn512m-Xmn32mdottyandfinagle-httpwith 60 s dumps lose no names in plain lazy mode. Their code still counts as unloadable (Renaissance uses a loader per benchmark), so the drain captured 9–14k methods and used none, at about 0.1–0.2% of a core.Additional Notes:
std::unordered_mapunder a spinlock;method_resolve_lazy_unreadablelanded in Add fjmethodid=lazy: profile HotSpot without jmethodIDs #868 separately (c2ccd610d). This branch adds the same counter, so it conflicts with Add fjmethodid=lazy: profile HotSpot without jmethodIDs #868's head until it is rebased.How to test the change?:
testDebug -Ptests=*Lazy*passes with capture off (6/6).DDPROF_CAPTURE_UNLOADING=3, 5/6 pass, includingLazyRedefinitionStressTest.LazyJmethodIdTest.framesOfUnloadedClassAreDroppedSafelyfails as expected: it asserts that a frame of an unloaded class gets no name, and the capture names it.utils/phase2-proto/bench3.sh(recovery),gcbench.sh(GC pauses),RenWrap.java(Renaissance). Usage is in the hand-over doc.For Datadog employees:
credentials of any kind, I've requested a security review (run the
dd:platform-security-reviewskill, or file a request via the PSEC review form).
bewairealso runs automatically on every PR.🤖 Generated with Claude Code