I'd like to take this on, scoped down to shader compilation time only. Before writing anything I want to check the scope is what you'd want, since this issue is broad enough that I think that's why it's sat for a while.

Most of the instrumentation already exists. crates/rustc_codegen_spirv/src/linker/mod.rs wraps 29 linker passes in sess.timer(...), which feeds rustc's SelfProfilerRef. So per-pass timing is already obtainable today with -Zself-profile — nothing captures, stores or compares it. That seems like the cheap win, rather than building a measurement layer from scratch.

There's demonstrated demand. Several recent changes are compile-performance work that had to be measured by hand and can't be regression-tested:

batch mem2reg to process all variables in a single pass #547 reports mem2reg going from 228s to 18s on a real shader
Fix slow compilation of large zero-initialized arrays; warn on large non-zero fills #586 fixes let arr = [0; 32 * 1024] compiling very slowly
Clippy linting is very slow #528 (clippy linting very slow), [Migrated] Optimize spirv-builder clean build wall time #112 (clean build wall time), Parallelize codegen #445 (parallelize codegen)

Corpus is already in-tree. The 6 example shaders (sky-shader, compute-shader, reduce, mouse-shader, shared, simplest-shader), plus 330 compiletests and 49 difftest crates to draw pathological cases from. I don't think we need new shader content to start.

Proposed scope

An xtask bench subcommand that compiles a fixed corpus under -Zself-profile and emits per-pass timings as JSON.
A CI job that reports deltas on PRs as information only, never as a build gate.
That's it for a first patch.

On the CI point specifically: CI runs on shared ubuntu-latest runners, and #2 is open because they're slow and variable. Wall-clock regression gating on that hardware would flap constantly and people would learn to ignore it. I'd rather report per-pass numbers and let a human read them than add a check that fails randomly. If the project would rather have hard gating, that needs dedicated runners first and is a bigger conversation.

Deliberately out of scope for this first pass

Runtime/GPU performance — different problem, different harness.
The VulkanShaderExamples suites in Use VulkanShaderExamples rust-gpu port as benchmarks #315 and Use VulkanShaderExamples rust-gpu port as difftests #316, since other people have expressed interest there.
A criterion or CodSpeed micro-benchmark over linker::link(). This is very doable — linker/test.rs already builds a real rustc_session::Session inside rustc_driver::catch_fatal_errors, which is the hard part of calling link() outside a compiler run — but I'd rather land the macro layer first and let it tell us which passes actually dominate, instead of guessing which ones to micro-benchmark.

Two questions before I start:

Is xtask the right home for this, or would you prefer it live somewhere else?
@LegNeato, you mentioned rustc-perf and CodSpeed when you filed this. Do you have a preference, or is starting with plain -Zself-profile output and adding a service later acceptable?

Add performance testing #49

Description

Activity

LegNeato commented on Nov 7, 2024

sachinsharma3191 commented on Sep 11, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions