Repository navigation
QA run · dashboards (6/10) · 5b14b7f3 · 2026-10-19 · 4 PASS / 2 PARTIAL / 0 FAIL / 0 BLOCKED / 4 NOT-RUN #24573
Closed as not planned
Description
Run record for the 18.0 release sweep, selector dashboards (resolved with node scripts/checklist-select.mjs area:dashboards --json), plan B (Tier 1 + Tier 2; Tier 3 deferred).
Environment fingerprint
- framework:
5b14b7f3ad8e2e0786d228772f473d07b36b2477(objectstack-ai/objectstack main), working tree clean - console:
.objectui-sha=4016370af3c9c9dafec2bc5299afd8e8079701b8, built withpnpm objectui:build;packages/console/dist/.objectui-shastamp equals the pin (pnpm check:console-shagreen), so the stale-console-bundle trap is ruled out for every console observation - app: showcase (examples/app-showcase), canonical boot
OS_PORT=PORT pnpm -C examples/app-showcase exec objectstack dev --ui --seed-admin -p PORT -d file:/tmp/qa18/ITEM/data.db; every hand-driven item had its own port and file DB (listed per item below); seed:--seed-admin(admin@objectos.ai) + showcase seed + per-item scratch fixtures - Tier 1 pins: vitest per package against the same tree (
pnpm exec vitest run FILES), 97 framework files / 2209 tests, 0 failed; objectui-side pins run inside a clone of objectui at4016370af3c9(unit vitest +playwright.live.config.tsagainst a showcase boot on :47431 through the objectui console dev server), RUNNER option (b) - browser: Playwright with
/opt/pw-browsers/chromium, own context per persona, sign-in through the form - timestamps: 2026-10-19, environment prep 12:26–12:43 UTC, testing 12:43–13:53 UTC; Tier 2 items were driven in parallel by sub-runners (≤ 3–4 browser items at once), each on its own port/DB
Scope
Selector dashboards → 10 runnable item(s). Blocked items hidden by the resolver were not run (no --include-blocked).
Tiering (checklist-test SKILL §1): Tier 1 = item carries automated.ref (pin run, cited); Tier 2 = unpinned P0/P1 (hand-driven); Tier 3 = unpinned P2 (deferred, not-run).
| item | priority | surface | tier | revision | item verdict |
|---|---|---|---|---|---|
dashboards.strict-widget-rejects-stray-keys |
P1 | mixed | 2 | 2 | pass |
dashboards.chart-first-paint |
P2 | browser | 3 | 2 | not-run |
dashboards.drill-through-range |
P2 | browser | 3 | 2 | not-run |
dashboards.chart-type-matrix |
P1 | browser | 2 | 5 | pass |
dashboards.dataset-report-authoring |
P1 | mixed | 1 | 2 | partial |
dashboards.empty-null-bucket-boundaries |
P2 | mixed | 1 | 1 | partial |
dashboards.global-filters-rescope |
P1 | browser | 2 | 4 | pass |
dashboards.system-overview-live-counts |
P2 | mixed | 3 | 3 | not-run |
dashboards.cube-query |
P2 | api | 3 | 4 | not-run |
dashboards.analytics-exposure-gate-and-measure-aggregate |
P1 | api | 1 | 1 | pass |
Per-clause verdicts
A = acceptance[n], N = negative[n]. Tier 1 rule: a clause is pass only when a passing pin covers it (server-side oracles, or clauses the ref names); clauses a ref says it does not cover, and UI-oracle clauses a server-side pin cannot see, are not-run. Evidence is text from the oracle — no images.
dashboards.strict-widget-rejects-stray-keys — pass (hand-driven (port 47481 · file:/tmp/qa18/stray/data.db))
| clause | verdict | evidence |
|---|---|---|
| A0 | pass | 11 legacy keys each PUT /api/v1/meta/dashboard/qa_stray_key_probe?mode=draft → 422 INVALID_METADATA at widgets.0 Unrecognized key(s) on this dashboard widget naming the key |
| A1 | pass | each legacy message carries the ADR-0021 prescription (bind a dataset, select dimensions + values) |
| A2 | pass | chartFlavour → 422 with no did-you-mean (none close); control dimension → Did you mean dimensions? |
| A3 | pass | chartConfig dataset/drillDown/aggregate/objectName → 422 at widgets.0.chartConfig with wrong-layer guidance |
| A4 | pass | after 18 rejected PUTs GET → 404; after a good draft + publish, rejected PUTs leave the body unchanged; /history lists only create + publish |
| A5 | pass | corrected widget renders 5 bars; its POST /analytics/dataset/query equals the direct query (5 rows) — equal bar heights noted as single-value weakness |
| N0–N1 | pass | no variant accepted; no stray-key save returned 2xx |
dashboards.chart-first-paint — not-run (Tier 3)
| clause | verdict | evidence |
|---|---|---|
| A0–A3, N0–N0 | not-run | Tier 3 deferred by the 18.0 release-sweep plan |
dashboards.drill-through-range — not-run (Tier 3)
| clause | verdict | evidence |
|---|---|---|
| A0–A4, N0–N0 | not-run | Tier 3 deferred by the 18.0 release-sweep plan |
dashboards.chart-type-matrix — pass (hand-driven (port 47482 · file:/tmp/qa18/matrix/data.db · wall clock 2026-10-19 13:30 UTC))
| clause | verdict | evidence |
|---|---|---|
| A0 | pass | ChartTypeSchema has exactly 20 members; all 20 rendered as their own geometry (16 families + 4 synonyms); 18 widget queries 200 |
| A1 | pass | mark counts equal server buckets: bar 5, column 4, horizontal-bar 5, pie 5, donut 4, funnel 5, radar radii 1:4:3:2, line/area 2 points, scatter 6 |
| A2 | pass | combo: 2 bars + 1 line on dual axes; tooltip 81.667 = API 81.67 |
| A3 | pass | treemap 5 rectangles by est_hours; sankey 6 nodes / 5 links; scatter 6 points |
| A4 | pass | kpi / gauge / solid-gauge / bullet + metric all show 10 Tasks from rows [{task_count:10}] |
| A5 | pass | pivot status×priority 8 cells and totals 1/4/3/2 = 10 match the aggregate; table widget 3 accounts × 3 measures equal the API |
| A6 | pass | vitest test/coverage.test.ts 13/13 incl. covers every distinctly-renderable ChartType |
| A7 | pass | compareTo previousPeriod: literal windows give 7 and 3; widget returns {invoice_count:7, invoice_count__compare:3}, tile 7 · ↑133%; bare-string spelling → 422 with the upgrade text |
| A8 | pass | vitest src/ui/dashboard-compareto.test.ts 17/17 |
| N0–N2 | pass | no family fell back; no blank widget; comparison present and correct |
dashboards.dataset-report-authoring — partial (Tier 1 pin)
| clause | verdict | evidence |
|---|---|---|
| A0, A1 | pass | pin: dashboard-designer-roundtrip.dogfood.test.ts 3/3 passed |
| A2 | not-run | screenshot oracle — the pin is server-side; UI half not driven in this Tier 1 pass |
| A3–A5 | pass | pin: dashboard-designer-roundtrip.dogfood.test.ts 3/3 passed |
| A6, A7 | not-run | screenshot oracle — the pin is server-side; UI half not driven in this Tier 1 pass |
| N0, N1 | not-run | negative spans the UI half not driven in this Tier 1 pass |
dashboards.empty-null-bucket-boundaries — partial (Tier 1 pin)
| clause | verdict | evidence |
|---|---|---|
| A0 | not-run | screenshot oracle — the pin is server-side; UI half not driven in this Tier 1 pass |
| A1–A4 | pass | pin: date-bucket-parity-conformance.test.ts 8/8 passed |
| A5 | not-run | screenshot oracle — the pin is server-side; UI half not driven in this Tier 1 pass |
| N0, N1 | not-run | negative spans the UI half not driven in this Tier 1 pass |
dashboards.global-filters-rescope — pass (hand-driven (port 47483 · file:/tmp/qa18/filters/data.db · scratch package com.qa18.filters))
| clause | verdict | evidence |
|---|---|---|
| A0 | pass | region EMEA → 9 re-queries; invoice widgets {region:emea}, account widgets {sales_region:emea}, each in $and with its own date field |
| A1 | pass | custom window 2026-04-01..06-30 → issued_on / signed_on bounds; invoices 2 and accounts 3 = REST seed |
| A2 | pass | kpi_accounts_alltime (filterBindings false) never re-queried; reads 14 throughout |
| A3 | pass | EMEA + last 90 days: Invoices 1 (REST: only INV-1010 in window); Accounts Signed 1 |
| A4 | pass | null signed_on rows excluded: 11 = 14 − 3 prospects |
| A5 | pass | Task Status In Review → 2 (REST total 2); Done → intersection 0; unbound KPIs unchanged |
| A6 | pass | Studio inspector filterBindings authored → draft PUT → publish-drafts 200; re-read shows the bindings; runtime honours them after reload |
| A7 | pass | pristine bar has no Reset; dirty bar shows Reset; Reset restores defaults with 9 re-queries |
| A8 | pass | static {value,label} options show the label and commit the value; optionsFrom issues a server GROUP BY (3 rows), complete |
| N0–N3 | pass | every repaint preceded by a re-query; opted-out KPI never drifted; binding persisted; option list not truncated |
dashboards.system-overview-live-counts — not-run (Tier 3)
| clause | verdict | evidence |
|---|---|---|
| A0–A5, N0–N3 | not-run | Tier 3 deferred by the 18.0 release-sweep plan |
dashboards.cube-query — not-run (Tier 3)
| clause | verdict | evidence |
|---|---|---|
| A0–A5, N0–N3 | not-run | Tier 3 deferred by the 18.0 release-sweep plan |
dashboards.analytics-exposure-gate-and-measure-aggregate — pass (Tier 1 pin)
| clause | verdict | evidence |
|---|---|---|
| A0–A3 | pass | pin: api-exposure-door.test.ts 30/30 passed; unexposed-definition-registration.test.ts 12/12 passed; dimension-label-exposure.test.ts 12/12 passed; preview-column-enrichment.test.ts 12/12 passed |
Reproduction rules
No fail in this selector.
Derived item verdicts
4 pass · 2 partial (Tier 1 pins cover the server-side clauses; the screenshot/DOM clauses of dashboards.dataset-report-authoring and dashboards.empty-null-bucket-boundaries were not driven in this Tier 1 pass) · 4 not-run (Tier 3).
Checklist-accuracy findings / fixture gaps (for the wave anchor card)
dashboards.strict-widget-rejects-stray-keyssteps[4] (Studio inspector door) cannot be executed: the widget inspector is a structured form with no free-key or JSON entry — reword as "confirm the inspector offers no free-key path". A pending dashboard draft is unreadable until publish (GET …?mode=draft→ 400 "mode query parameter is not supported") — name the publish step.dashboards.chart-type-matrixsteps[5] is stale: the combo widget no longer authorsseries[](removed by the semantic entrydashboard-widget-chart-config-structure-refused; the renderer picks each measure's mark) — re-point the step. acceptance[4] wording "identical to the metric tile" should read "same value" (the metric card's chrome differs). Seed is thin for trend charts (2 month buckets: single-datapoint trap, noted).dashboards.global-filters-rescopefixtures.requires[2] is incomplete: the Studio interface pillar needs the scratch package to also carry an app whose navigation points at the dashboard (PUT /api/v1/meta/app/NAME?package=PKG) — name both. Cosmetic: the inspector labels the date binding row with the raw namedateRange.
Hand-off (items without a judged verdict)
dashboards.chart-first-paint,dashboards.drill-through-range,dashboards.system-overview-live-counts,dashboards.cube-query(all P2) — not-run: Tier 3 deferred by the 18.0 release-sweep plan.
Generated by Claude Code
Activity
objectstack-fleet commentedon Oct 19, 2026
Parked-on: #24579 · this record's checklist-accuracy findings, fixture gaps, partial coverage gaps and not-run handover (wave anchor §2–§4)
Generated by Claude Code
objectstack-fleet commentedon Oct 19, 2026
Superseded-by: #24612
Round 2 at ba4afcc re-judged this record's A-class unjudged clauses (both dashboards partials now PASS on their UI half) and carried every other verdict forward to #24612 (
Carried-forward:lines there, valid for 5b14b7f). B-class gaps stay parked on #24579.Generated by Claude Code