Skip to content

[P0] Metadata property liveness audit: ~half of all spec properties are dead; a cluster of security props is parsed-but-unenforced #1878

Closed
5 of 5 issues completed

Description

@xuyushun441-sys

Summary

A protocol-level audit of every metadata type cross-referenced each schema property's spec definition against its actual consumers (file:line) in the runtime + renderers, to determine (1) does it take effect? and (2) is it necessary/reasonable? Liveness = code consumption, not browser observation.

Result: roughly half of all metadata properties are DEAD — defined in the protocol but read by neither the runtime nor the renderers, so authoring them is a silent no-op. The full evidence (17 per-type reports + cross-type synthesis, each with file:line) is in PR #1864 → docs/audits/ (README.md is the index).

This issue is the umbrella; the checklists below are scoped so they can be split into separate issues/ADRs.


🔴 P0 — Parsed-but-UNENFORCED security props (latent access-control gaps)

Properties that imply a security boundary but enforce nothing. Either enforce or remove — shipping them as-is is a compliance/security liability.

  • PolicySchema is 100% dead — password complexity, session idle/absolute timeout, forceMfa, IP allow-list (trustedRanges/vpnRequired), audit retention/redaction all parsed, none enforced; not even registered as a metadata type. better-auth (auth-manager.ts:458) runs hardcoded defaults instead → false sense of compliance.
  • Permission lifecycle ops ungated — allowTransfer / allowRestore / allowPurge omitted from OPERATION_TO_PERMISSION (permission-evaluator.ts:8-16); ownership transfer, undelete, and hard-delete/GDPR purge are not gated by RBAC.
  • Agent access control is a no-op — agent.permissions / visibility / access never enforced; the chat route hardcodes ['ai:chat','ai:agents'] (agent-routes.ts:109). "Who can chat with this agent" enforces nothing.
  • Flow runAs never switches execution identity (service-automation engine ignores it) — a flow declaring runAs: system does not elevate/de-elevate.
  • Object apiEnabled / apiMethods not enforced by REST — an object cannot actually be hidden from the API via these flags.
  • Action disabled (CEL form) silently ignored — renderers read a non-spec enabled (action-button.tsx:56); the predicate form of disabled has zero consumers.
  • Role parent dead (team-graph.ts:27 does not walk it) — the documented "managers see subordinates' data" rollup is unimplemented.
  • SharingRuleSchema disconnected from the live engine — runtime enforces a divergent sys_sharing_rule model (JSON criteria_json, recipient enum user/team/department/role/queue); the spec's CEL condition + role_and_subordinates/guest recipients are never compiled (unparsable CEL → "match nothing").

🔴 P1 — Finish the ADR-0021 analytics migration

Spec moved to dataset+values/dimensions, but renderers/Studio still read the removed legacy shape:

  • Chart view variant dead-on-arrival — ListView.tsx:1413 / ObjectView.tsx:754 read legacy xAxisField/yAxisFields/aggregation; spec exposes dataset/dimensions/values → a spec-compliant chart renders nothing.
  • Dashboard renderer + Studio WidgetConfigPanel still author/read legacy object/categoryField/valueField/aggregate while dashboard.zod.ts:153 declares dataset+values required → Studio emits widgets the spec rejects.
  • Report chart dead on the dataset path; ReportColumnSchema/ReportGroupingSchema/ReportChartSchema are obsolete type-only re-exports.
  • (Same debt that invalidated the showcase dashboard/report seeds.)

🟠 P2 — Spec hygiene

  • Naming drift → silent no-ops: field maxLength/minLength/referenceFilters/maxRating; page type→pageType & label→title & visibility; dashboard title vs label; app reads accentColor/badgeVariant/separator not declared in spec; flow http vs deprecated http_request (Studio authors the alias); skill requiredPermissions vs permissions; agent knowledge.{topics→sources}; webhook object→object_name/isActive→active.
  • Write-only / disconnected metadata: tool metadata is a one-way projection (can't author a working tool as metadata — no implementation/handler field, no executor loads it); EmailTemplateSchema is registered but the runtime reads a differently-shaped sys_email_template; the spec-bridge page/dashboard bridges are orphaned (their exclusive props dead).
  • Aspirational config (rich spec, zero runtime) — prune or mark experimental: field enhanced-type configs (barcode/qr/slider/rating/color/location) + governance (encryption/masking/audit/dataQuality); object enable/versioning/partitioning/cdc/softDelete/search; agent autonomy (memory/guardrails/structuredOutput/lifecycle, except planning.maxIterations); tool outputSchema; job retryPolicy/timeout; theme spacing/breakpoints/logo/rtl/density; translation messageFormat:'icu' (no ICU engine), supportedLocales, cache; PortalSchema (entire — not registered, no route, no renderer).
  • Renderer reads UNDECLARED props (inverse drift) — dashboard component/data/rowField/columnField; view ObjectView form-adapter keys; app accentColor/badgeVariant/separator. These break a strict Schema.parse().
  • Designer authoring gaps (live prop, no Studio editor) — dataset filter/format/derived/dateGranularity; skill triggerConditions (the activation-critical field); flow notify (absent from the static palette).

Healthiest vs worst (from the audit)

  • Healthiest (near-total liveness): hook, validation.
  • Worst: policy (100% dead), portal (100% dead), tool (write-only).

Evidence + per-property recommendations: PR #1864, docs/audits/*.md.

🤖 Generated with Claude Code

Activity

  1. os-zhuang commented on Jun 15, 2026

    @os-zhuang
    Contributor

    Split into sub-issues

    This umbrella has been decomposed into tracked sub-issues (each links back here, labeled by priority):

    🔴 P0 — security enforcement (priority:p0 + security)

    🔴 P1 — analytics (priority:p1)

    🟠 P2 — spec hygiene (priority:p2)

    The audit's suggested 3-ADR grouping maps to: Security enforcement (the P0 set), Finish ADR-0021 (#1890), Spec hygiene (the P2 set).

  2. os-zhuang commented on Jun 15, 2026

    @os-zhuang
    Contributor

    Staged plan for the P0 security cluster — ADR-0049

    Decision recorded in ADR-0049 (docs/adr/0049-no-unenforced-security-properties.md): a spec property naming a security boundary must be enforced, explicitly experimental, or absent — never parsed-but-silently-unenforced (that's a false sense of compliance).

    Given the platform's pre-MVP stage, the cluster splits by whether the feature already exists — we do not build enforcement for features that don't exist yet:

    Disposition Items Action now
    Feature absent → mark experimental / remove #1882 Policy · #1883 lifecycle bits · #1887 SharingRule (spec form) · #1888 flow runAs stop the spec from promising; re-introduce with the feature at M2
    Feature live, gate missing → enforce #1884 agent access · #1889 object apiEnabled · #1885 action disabled localized route/renderer check now
    No-regret correctness fix evaluator fail-open fail CLOSED for destructive ops

    PR A (no-regret, shipped on branch fix/p0-unenforced-security-props-1878)

    PR B (next) — real gaps where the feature is live

    Deferred to M2/production (heavy, feature + enforcement together)

  3. os-zhuang commented on Jun 15, 2026

    @os-zhuang
    Contributor
  4. os-zhuang commented on Jun 16, 2026

    @os-zhuang
    Contributor

    Closing the umbrella — audit delivered, disposition fully split out

    The audit this issue tracked is done and has been operationalized into a standing system, and every checklist item has been split into its own labeled, independently-tracked sub-issue. Keeping this umbrella open now only duplicates that tracking, so closing it.

    What the audit became (standing system, not a one-off)

    • Spec-liveness gate — packages/spec/liveness/*.json ledgers + packages/spec/scripts/liveness/check-liveness.mts, wired as the blocking Spec property liveness CI check. Every authorable property of 10 governed types is classified live/experimental/dead with evidence; the ratchet blocks new undeclared surface. This is the living source of truth that replaces the static audit docs.
    • Cross-repo correctness — the original audit missed objectui's metadata-admin authoring layer; a cross-repo re-audit corrected ~25 props dead→live (fix(spec): cross-repo liveness re-audit — 16 props dead→live (objectui authoring UI) #1956 et al.). A monthly drift re-audit cron now catches dead→live drift the completeness gate can't.
    • Author-side loop closed (feat(cli): liveness author-warning lint — close the spec-liveness loop #1966) — objectstack compile now warns when an author/AI sets a misleading dead/experimental property (enable.feeds, field.columnName, …) with a corrective hint. Measurement → gate → drift-cron → author-prevention.

    Shipped against the checklist

    Remaining items — each tracked in its own open issue (nothing orphaned)

    Closing as completed (deliverable shipped + operationalized); execution continues in the sub-issues above.

  5. 25 remaining items

  6. os-zhuang commented on Jul 27, 2026

    @os-zhuang
    Contributor

    SharingRule loose end from the recheck (#3503) closed by #3557: authoring ShareRecipientType reconciled with the enforced runtime — group renamed → team (wired to the live TeamGraphService expansion), business_unit added, guest pruned (anonymous access = public-form grant / share links), and the never-enforced owner-type rules removed (return if membership-reactive re-materialisation is designed). Every authorable recipient/rule type now enforces (ADR-0078); proven on a real boot — the migrated showcase rule seeds (ruleCount 4→5) with criteria_json {done:false}. Remaining from the recheck: AuditConfig-cluster prune decision (needs an owner call on dead-vs-roadmap) and per-org IP allow-list (#2571).

  7. os-zhuang commented on Jul 27, 2026

    @os-zhuang
    Contributor

    2026-07 dead-cluster sweep — disposition record (8 PRs)

    Follow-through of the recheck above: the report slice, the security-props cluster, and three chained dead-config clusters are now fully dispositioned. Recording here so triage of the remaining audit sections doesn't re-excavate.

    Cluster PRs Disposition
    Report dead exports (ReportColumn/ReportGrouping + objectui SpecReport* re-exports, chart.groupBy, legacy ReportViewer chart fallback) #3488 / objectui#2816 (#3463) removed; live path = dataset-bound DatasetReportRenderer
    Report aria/performance + chart dogfood + stale audit refresh #3496 pruned; report.chart confirmed live (audit's "dead" claim corrected); showcase chart report added
    Security-props recheck + ledger fix #3503, #3506 cluster largely closed (see comment above); flow-run-as reclassified enforced
    AuditConfig cluster (system/audit.zod.ts, whole module) #3593 removed — live audit = plugin-audit always-on capture + trackHistory + lifecycle audit retention + settings overrides
    Static capabilities descriptor (ObjectQL/ObjectUI/Kernel/ObjectStack/ObjectOS CapabilitiesSchema) #3605 / objectui#2860 removed — defaults contradicted the live platform (FLS/RLS/audit default(false) while enforced); live discovery = GET /api/v1/discovery (declared===enforced) + /.well-known
    Orphaned FeatureFlagSchema module (kernel/feature.zod.ts) #3630 removed — protocol home died with #3605; live toggles = feature_flags settings manifest + PUBLIC_AUTH_FEATURES

    Pattern note for the remaining sections (§3 naming drift, §4 aspirational, §5 write-only): these clusters chain — each removal exposed the next orphan (AuditConfig → auditLogging boolean → capabilities descriptor → FeatureFlagSchema). Worth pulling each thread to its end rather than pruning single symbols. All removals shipped as minor per the launch-window policy, each changeset carrying a dead-knob → live-surface migration table.

    Next confirmed actionable from the recheck: the objectui action.disabled wire-rename (spec declares disabled CEL; the renderer reads non-spec enabled — authored predicates silently no-op).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationpriority:p0Critical: blocker, must ship before MVPsecurity

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions