[abrichr]

Adds packages/flow/benchmark/lifecycle/, a harness that measures whether OpenAdapt removes work under ordinary business conditions on a real local OpenEMR 8.0.0.3 with synthetic data.

Draft: paused by the coordinator before the soak. No soak report is committed yet.

What it has

• The metric: verified completed items per operator-hour, computed only from the run's ledger. Its parts are reported beside it:
  • setup, per-item operator and maintenance minutes;
  • false-stop rate and silent-wrong rate, both with Wilson 95% CIs;
  • wall time and host load;
  • model calls and cost.
  • Two comparison views: the headline if every silent wrong were later found and fixed, and an all-manual baseline.
• The cost model: versioned in cost_model.py, shown at x0.5, x1 and x2. No real operator timings exist yet. Desktop's request and answer timestamps were all produced by scripts, and the calibration note says so.
• The workload: a seeded referral stream with 13 exception kinds and 4 server-side UI changes. Every event is logged in the ledger with exactly what was changed.
• The oracle: an independent SQL read from inside the database container.
• The operator: a scripted, deterministic operator charged from the cost model.
• The arm interface: lifecycle-arm/1. The baselines session's arms build on it; see branch claude/lifecycle-arms.
• The OpenAdapt arm:
  • governed mode is the Standard governed path that the certified referral runs use.
  • cli mode runs one openadapt-flow run per item, attached to a signed-in browser over CDP.
• Tests: an in-memory app and two toy arms. tests/test_lifecycle_benchmark.py passes 13 of 13 locally.

Smoke findings on real OpenEMR

These are a handful of items each, so they are not quotable rates.

• openadapt-flow run cannot run the certified referral program.
  • With --profile standard it stops with: "Standard and Regulated actuation requires a signed qualification admission."
  • With no profile, the encryption gate refuses an unencrypted bundle. openadapt-flow seal then refuses the certified bundle with: "governed authorization template cannot be reproduced from the sealed certification".
  • With --allow-unencrypted and an attached signed-in browser, every item halts at step_000 (4 of 4): the generic web backend fails the structured identity check on the title field.
  • When attached over CDP, auto-repair turns itself off with: "an attached browser can't be reopened for a probe".
• On the governed path, verified items take about 28 s each.
  • Both runs stopped in the duplicate check (step_039) for a new patient who shares a last name with an existing chart. OpenEMR's duplicate search lists that chart, and the program stops on any hit. The oracle counts this as a false stop.
  • On the governed path, no per-item approval gate is wired, no repair agent is wired, and pipelined read-back is refused ("durable runs verify inline").

Not done

• The full soak (300 items) and its committed report.
• The 40-item comparison stream shared with the baselines arms.
• openIMIS.
• CAPABILITIES.md row.

🤖 Generated with Claude Code