Skip to content

feat(benchmark): add clock resolution estimation iteration limits and CLI config - #3209

Open
Tyagiquamar wants to merge 5 commits into
catchorg:develfrom
Tyagiquamar:fix/benchmark-clock-resolution-limit
Open

Tyagiquamar wants to merge 5 commits into
catchorg:develfrom
Tyagiquamar:fix/benchmark-clock-resolution-limit

Conversation

@Tyagiquamar

Copy link
Copy Markdown

Problem Description

Resolves #3180.

When running benchmarks in constrained environments (such as embedded targets with limited RAM, e.g. 16MB), estimate_clock_resolution could double iterations repeatedly in run_for_at_least until reaching excessive counts (e.g. 1,280,000+ points), causing std::bad_alloc due to concurrent allocations of times and deltas vectors. Additionally, clock resolution estimation time was hardcoded at 500ms without command-line configurability.

Solution

  1. Added max_iterations bound support to run_for_at_least.
  2. Added clock_resolution_estimation_max_iterations cap in estimate_clock_resolution and bounded sample count allocation in estimate_clock_cost.
  3. Exposed --benchmark-estimate-clock-time on the command-line interface, ConfigData, Config, and IConfig (default: 500ms) matching --benchmark-warmup-time.
  4. Added unit tests for clock estimation iteration limits in InternalBenchmark.tests.cpp.
  5. Added CLI parsing test cases in CmdLine.tests.cpp.

Tests

  • Unit tests in InternalBenchmark.tests.cpp
  • CLI configuration tests in CmdLine.tests.cpp

@Tyagiquamar

Copy link
Copy Markdown
Author

Fixed the mixed-duration std::max call by converting elapsed nanoseconds to FDuration before comparison. This preserves the one-nanosecond lower bound and fractional arithmetic. Docker Ubuntu 22.04/GCC 11.4/C++14 reproduced the original compiler error; the library and SelfTest build now pass. Focused warmup, resolution, estimate_clock_resolution and Benchmark Fibonacci checks passed (19 assertions, four cases, five samples per benchmark). clang-format 14 check on the changed block and whitespace checks passed. MSVC validation is left to CI.

The prior baseline refresh re-encoded StringManip toLower samples as UTF-8
and misplaced estimate_clock_resolution/with iteration limit. Restore the
latin-1 bytes AppVeyor emits and place the section next to its parent.
@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.19%. Comparing base (222e233) to head (6fb6461).
⚠️ Report is 11 commits behind head on devel.

Additional details and impacted files
@@            Coverage Diff             @@
##            devel    #3209      +/-   ##
==========================================
+ Coverage   91.17%   91.19%   +0.02%     
==========================================
  Files         206      205       -1     
  Lines        9080     9104      +24     
==========================================
+ Hits         8278     8302      +24     
  Misses        802      802              
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clock resolution estimation iterations limit

1 participant