[spec] FieldReferenceSchema is declared in the $between endpoints but NO backend resolves a $field inside a list

Measured while implementing #5222 (cross-field push-down). Filing unassigned — recording, not claiming.

The fact

packages/spec/src/data/filter.zod.ts declares FieldReferenceSchema in both $between endpoints, in the documentation copy and in the ENFORCED copy:

$between: z.tuple([
  z.union([z.number(), z.date(), z.string(), FieldReferenceSchema]),
  z.union([z.number(), z.date(), z.string(), FieldReferenceSchema])
]).optional(),

$in / $nin are z.array(z.any()), so a reference is admitted there too.

No backend resolves it. The in-memory evaluator — the one implementation the repo has for $field — does not, and this is structural rather than an oversight. matches-filter.ts resolveValue returns early for an array:

function resolveValue(raw: unknown, record) {
  if (raw && typeof raw === 'object' && !Array.isArray(raw) && '$field' in raw) { … }
  return raw;   // ← an ARRAY comes back unchanged, members unresolved
}

evalOp resolves the whole comparand, never the members. So for $in/$nin the raw reference OBJECT is compared with looseEq against a stored value (never equal), and for $between the raw object becomes an ordering bound.

Why it matters

This is the ADR-0049 enforce-or-remove shape at a declared position: the spec promises a comparand form that nothing implements, and the failure is SILENT on the memory path — { amount: { $in: [{ $field: 'budget' }] } } matches nothing and reports nothing. On an RLS check that is a denied write with no diagnostic; the $nin direction loses an exclusion the author wrote.

driver-sql / driver-sqlite-wasm refuse these positions loudly (INVALID_FILTER / 400) — #5041 installed that and #5222 deliberately kept it, precisely because there is no correct in-memory semantics for SQL to be equivalent to. So today the declaration is honoured by nobody and refused by two backends.

The decision this needs

Either:

Remove FieldReferenceSchema from the $between endpoint unions (and rule $in/$nin members out explicitly), making declared = enforced; or
Implement member resolution in matches-filter.ts and then decide whether the SQL faces can compile it conformance-equivalently (per-member OR-expansion, whose NULL and type-affinity behaviour is the open question [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 declined to guess at).

Not decided here — it is a contract-face call, same class as the #5222 rulings.

Refs

[spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 (cross-field push-down; refuses these positions, with the reasoning recorded in cross-field-conformance-cases.ts)
[spec] $field 跨字段比较:spec 声明 + cel-to-filter 产出,但无任何 SQL 执行层实现 —— enforce-or-remove 裁决位 #5041 (the original refusal + probe evidence)
ADR-0049 (enforce-or-remove)

Triage: needs-user-decision appended; domain:spec (filer-applied) confirmed correct.

Classification: the card itself frames the fork — remove FieldReferenceSchema from the $between endpoints (declared = enforced) vs implement member resolution — and states "Not decided here — it is a contract-face call". Either branch changes the acceptance surface of packages/spec, which is exactly the domain:spec criterion (standing instruction ③ reverse red line), and the choice is a maintainer contract ruling, same class as the [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 rulings.
Premise verified on origin/main @ 744b8f5: packages/spec/src/data/filter.zod.ts:319-320 still declares FieldReferenceSchema in both $between endpoints, and packages/formula/src/matches-filter.ts:342 still early-returns arrays unresolved (!Array.isArray(raw) guard). Both anchors hold.
Dedup: [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 (in flight, pm:dispatched) deliberately kept the SQL-side refusal and did not touch these positions; [spec] $field 跨字段比较:spec 声明 + cel-to-filter 产出,但无任何 SQL 执行层实现 —— enforce-or-remove 裁决位 #5041 is closed history. Sibling cards [spec] parseFilterAST lowers ['a', '=', { $field: 'b' }] to a bare field spec no backend reads as equality #7597/[spec] service-analytics' read-scope / Cube filter compilers still refuse $field, so a CEL field-to-field RLS rule 400s on those faces #7598 filed in the same batch cover the lowering sink and the analytics faces respectively — three distinct faces, no overlap in fix site. No convergence needed.
Release read: the $field capability itself is still landing via [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 (its corpus is not yet on main — zero CROSS_FIELD_CASES hits @ 744b8f5), and the SQL faces refuse these positions loudly, so this does not block the current RC on its own; the ruling can also fold into [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222's follow-up scope.

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

Generated by Claude Code

Maintainer ruling recorded 2026-08-11 (PM session, executing the maintainer's direct instruction in chat — verbatim: 「接受你的全部建议，请更新 issue 的状态和标签」, accepting the four-lens decision-inbox review in full).

Ruling: Remove. Take FieldReferenceSchema out of both $between endpoint unions, and rule $in/$nin members out explicitly — declared = enforced via removal (ADR-0049). ⛔ No member-resolution implementation without measured demand (startup-focus): zero consumers today, and two backends already refuse these positions loudly.

State: needs-user-decision → pm:queue (domain:spec seat).

Generated by Claude Code

Round-1 selection note (spec lane, session session_01JY2Q5Xto1u8YHADgrZDTnk): serialized BEHIND #7597 — same file (packages/spec/src/data/filter.zod.ts), strict cross-round serialization, and #7597's ruled silent-zero-rows fix goes first.

Facts recorded for the dispatching round (re-verified 09:18Z on origin/main):

FieldReferenceSchema still sits in both $between endpoint unions at :319-320 (documentation copy) and :819-820 (enforced copy) — the removal targets the LIST positions only; the $gt/$gte/$lt/$lte references (:197-209, :801-804) STAY, they are the [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 capability.
Dispatch only after [spec] parseFilterAST lowers ['a', '=', { $field: 'b' }] to a bare field spec no backend reads as equality #7597's PR merges; re-verify these anchors on the post-merge ref, and re-price this card with [spec] parseFilterAST lowers ['a', '=', { $field: 'b' }] to a bare field spec no backend reads as equality #7597's dev's answer on whether the lowering change moves the removal scope.

Generated by Claude Code

Claim: PM loop round 2 (spec lane — hot-file queue head cleared: #7597's PR #7672 MERGED 10:51Z)
Session: session_01JY2Q5Xto1u8YHADgrZDTnk
Branch: claude/issue-7596-between-list-fieldref-removal
Worktree: cloud dev session (own container)
Domain: domain:spec
File surface: packages/spec/src/data/filter.zod.ts — the $between endpoint unions ONLY (post-#7672 anchors: :319-320 documentation copy, :823-824 enforced copy — the enforced copy shifted +4 today) + explicit $in/$nin member rule-out + the cross-field conformance corpus's refusal arm + pins. ⛔ Not parseFilterAST/convertComparison — #7672 just landed there; its regions are settled. Stop on breach; explain in the report.
Container & model: M (ADR-0049 removal on the acceptance face with conformance obligations), mode:cloud, model: opus
Serial constraints cleared: predecessor #7597 (same file) MERGED as #7672 at 10:51Z — its dev's re-pricing answer: this card is 「untouched — neither easier nor harder, and still necessary」, and post-#7672 「the declared-but-unexecutable $field positions remaining are exactly the list ones this card removes」. No other in-flight writer on filter.zod.ts (queue + open PRs checked 10:52Z). Same-day churn: this file merged #7593 AND #7672 today — every anchor re-verified against the current origin/main (6a9dec6).

Generated by Claude Code

OS-DEV-REPORT — #7596

{
  "issue": 7596,
  "status": "done",
  "branch": "claude/issue-7596-between-list-fieldref-removal",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/7713",
  "premise_still_valid": true,
  "summary": "Executed the 08:08:22Z ruling as written: FieldReferenceSchema is out of BOTH $between endpoint unions (documentation copy and enforced copy, from one shared rangeEndpointSchema(index) factory so the pair cannot drift), and $in/$nin rule the { $field } member out explicitly via a superRefine on setMembershipSchema(op) rather than a type change — members stay z.any() because a membership list is heterogeneous and this schema is field-agnostic. Both refusals are message-bearing from one builder: the message names the position and index, prescribes the scalar-comparison alternative (#5222's shipped capability), and cites the 2026-08-11 ruling; it deliberately drops the SQL drivers' second escape ('or evaluate the rule in memory'), because at a list position the memory path is the one that answers with a wrong row set instead of an error. Measured on zod 4.4.3 that a tuple-level superRefine does NOT run once an element fails, so the $between refusal rides on the union's own error callback — which also leaves already-invalid endpoint shapes with zod's own code/path/wording, pinned as its own test. Whole-repo sweep found exactly two pins asserting the old acceptance (filter.test.ts:244 and :254); both flipped in place. matches-filter.ts, parseFilterAST/convertComparison and the equality-lowering file were not touched. Premise re-verified on 6a9dec6: both anchors hold at :318-321 and :822-825 as the claim comment predicted, and the evaluator's structural early-return is at matches-filter.ts:382-387 (it moved from the :342 the issue cited; the code is byte-identical).",
  "tests": "pnpm --filter @objectstack/spec test -> 'Test Files 377 passed (377) / Tests 9892 passed (9892)'. pnpm --filter @objectstack/spec typecheck -> tsc + check:scripts-typecheck + check:test-typecheck all OK ('57 file(s) / 265 error(s) held in test-typecheck-debt.json', unchanged). pnpm --filter @objectstack/spec check:generated -> 'All 13 generated artifacts are up to date' after regenerating gen:api-surface + gen:docs (only content/docs/references/data/filter.mdx moved: the two new describe() strings on $in/$nin; api-surface/ did NOT move; check:authorable-surface green with no anchor movement). pnpm check:driver-conformance / check:merge-driver / check:adr-anchors / check:spec-parsed-alias -> all exit 0. Driver + consumer suites: driver-sql 'Test Files 86 passed | 4 skipped (90) / Tests 1399 passed | 50 skipped', driver-sqlite-wasm '24 passed / 374 passed', objectql '182 passed / 3228 passed', service-analytics '71 passed / 1442 passed'. Downstream typecheck sweep over every package with $between in source (driver-memory, driver-mongodb, driver-sql, driver-turso, formula, objectql — all 'Done'); this is a PREFIX/consumer-direction selection by hand (the packages that consume the changed declaration), not a pnpm '...pkg' filter. REVERSE VERIFICATION, direction predicted before running: RED. Removed the fix with `git checkout origin/main -- packages/spec/src/data/filter.zod.ts` (no git stash), predicting exactly 9 failures — 5 in the $between block, 4 in the $in/$nin block, with 'does not repaint the refusals that were already there', 'leaves every other member shape open' and the #5222 ordering-slot control staying green. Result: 'Tests 9 failed | 119 passed (128)', the predicted 9 by name. Restored with `git checkout claude/issue-7596-between-list-fieldref-removal -- ...` -> '128 passed'. CI on PR #7713: in_progress at report time — not polled, per the report-at-draft-PR-time contract.",
  "open_questions": [
    {
      "question": "Bump level for a change that narrows an acceptance surface, and the ADR-0087 conversion question the Check Changeset gate asks.",
      "options": [
        "A: @objectstack/spec major, ADR-0087 conversion answered not-required in writing inside the changeset",
        "B: minor, on the launch-window 'ship breaking changes as minor' convention",
        "C: major plus an ADR-0087 D2 conversion rewriting a { $field } endpoint into { $gte: ref, $lte: ref }"
      ],
      "recommendation": "A, taken. major because declared shapes stop parsing; the repo is in changesets pre-mode (.changeset/pre.json, tag rc) where check-changeset-no-major stands aside by design and 148 pending changesets already carry major, so B would understate the change for no gate benefit. C is rejected on ADR-0087 D2's own losslessness requirement, not on convenience: no key is retired ($in/$nin/$between all remain with their arity intact, so there is no new shape to convert TO), a { $field } endpoint has no literal equivalent, and rewriting it to two ordering bounds would not preserve behaviour but invent it — the removed shape evaluated to 'matches nothing' in memory and to a 400 on both SQL faces, so any conversion producing rows changes every existing answer. Blast radius measured rather than asserted: the whole-repo sweep found zero templates, seeds, example apps or stored metadata carrying the shape."
    },
    {
      "question": "Should the four driver-door refusal cases in cross-field-conformance-cases.ts be retired now that the schema refuses these positions?",
      "options": [
        "A: keep all four verbatim, defence in depth",
        "B: retire them as now-unreachable"
      ],
      "recommendation": "A, taken. The schema door is not on every path to a driver — no face re-validates a find() `where` against FieldOperatorsSchema (see the NormalizedFilterSchema finding below), and a permission filter assembled in code never passes it at all. A driver that trusted the declaration would answer these shapes with the silent zero-row result #5041 found here in the first place. The corpus note now records this reasoning instead of the old 'filed as its own finding' line."
    }
  ],
  "out_of_scope_findings": [
    "filed as #7711: NormalizedFilterSchema accepts ANY field-condition shape — its $and/$or/$not union's second branch is a non-strict all-optional object, so it catches whatever the FieldOperatorsSchema record branch rejects. Measured identically on origin/main and on this branch, with a clean control that has nothing to do with #7596 ({ $and: [{ c: { $null: 'not-a-boolean' } }] } parses green while FieldOperatorsSchema refuses the same operator map). Observation-class: no runtime .parse caller exists outside packages/spec's own tests, so nothing a user hits today — labelled `finding` + `domain:spec`, no pm:queue. This PR pins the behaviour with that control rather than fabricating a red assertion for it."
  ]
}

Notes for the PM, outside the JSON:

No skip-changeset label — this PR ships .changeset/filter-list-field-reference-removed.md, so the label would be wrong.
Region boundary respected: parseFilterAST / convertComparison and filter-field-reference-lowering.test.ts (fix(spec): lower equality triples with a $field comparand to {$eq: ref} (#7597) #7672's settled regions) are untouched, as is matches-filter.ts:382 — the evaluator's array early-return was read as context and left alone.
os-regen relay: one tracked generated file moved, content/docs/references/data/filter.mdx. check:generated is green on the branch.

Generated by Claude Code

chore(changeset): answer the ADR-0087 disposition question with the m…

Addendum to the report above — one CI patch round on PR #7713, handled; the JSON report is otherwise unchanged.

Check Changeset went red on 1b487ac: check-adr-0087-registration.mjs (#6148) requires the ADR-0087 question to be answered in its machine-readable marker, not only in prose. The changeset argued not-required at length but carried no adr-0087: line, so the gate could not read the answer.

Fixed in ed90574 by adding the disposition in the closed vocabulary:

not-required (no-migration-prescription) — no key is retired ($in / $nin / $between all remain with their arity intact, so there is no old shape to convert to a new one); no lossless transform exists, because a { $field } endpoint has no literal equivalent and rewriting it to two ordering bounds would invent behaviour rather than preserve it; and the sweep measured zero consumers. Nothing mechanical to prescribe, and nothing to prescribe it to.

Verified locally against origin/main before pushing — the whole Check Changeset job, not just the failing step:

check-adr-0087-registration  exit 0  → "1 declared-breaking changeset(s), each carrying an ADR-0087 disposition"
check-empty-changeset        exit 0
check-changeset-no-major     exit 0  → pre-release mode (tag: rc), guard stands aside as designed
check-changeset-fixed        exit 0  → fixed group in sync with 69 public packages

Note for the next spec-lane card: this gate reads the changeset from the diff against the base, not from the working tree, so a marker added but not committed still reports as missing. That cost one local lap here.

No source or test change in ed90574 — the changeset body only. Not polling CI further; the PM owns convergence.

Generated by Claude Code

chore(regen): regenerate content/docs/references after merging origin…

os-regen pre-enqueue sync lap for PR #7713 — done, pushed, CI restarted. Evidence below; not waiting on it.

Merge


merged	`origin/main` @ `9d4dfc4` (feat(spec): submitBehavior.url — #7496 / #7657)
merge commit	`61d6a29` — clean, no conflicts, 90 files / 7583 insertions
method	`git merge origin/main`; no rebase, no force-push
branch head	`fedf5e1`

Worktree was the existing per-task one, clean at ed90574 — reused, no --force anywhere.

os-regen routing (read live from `.gitattributes`, 12 routed paths)

All 12 reset to origin/main before regeneration. Only one of them differed after the merge: content/docs/references/data/filter.mdx — this branch's own generated change, which the regeneration then rebuilt. Committed as f980954 (reset) and fedf5e1 (regen). The tree was not in MERGE state when gen:schema ran — asserted explicitly on MERGE_HEAD before building, per the #5370 trap.

pnpm --filter @objectstack/spec check:generated
✓ All 13 generated artifacts are up to date.

Survival assertions

(a) My own. The two $in / $nin rows are back in filter.mdx after regeneration (content/docs/references/data/filter.mdx:122-123), each carrying the #7596 citation; $field no longer appears in any $between position, and the FieldReference schema section itself is untouched at :56-62 (the scalar capability is not what this card removed). Refusal pins green: pnpm --filter @objectstack/spec exec vitest run src/data/filter.test.ts → Test Files 1 passed / Tests 128 passed.

(b) Siblings.

sibling	assertion	result
#7657	`content/docs/references/ui/view.mdx:286` — `submitBehavior` row with the ruled headline	present: "…`url` is relative-only and interpolates only declared record fields as `{{record.field_name}}`, URL-escaped (ruled 2026-08-11, #7496)"
#7657	`content/docs/ui/forms.mdx` §"What a `redirect` `url` may be"	present at `:401`, with all three numbered rules and the accepted/refused examples at `:411-416`
#7655	tenant/environment plan fields still `string`	`content/docs/references/cloud/environment.mdx:60,200,236` and `cloud/tenant.mdx:75,105,124` — all `string`, the opaque widening intact

The strongest form of that check, run over the whole routed set rather than three spot-checks:

git diff origin/main --stat -- <all 12 os-regen-routed paths>
 content/docs/references/data/filter.mdx | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Every sibling's generated content is byte-identical to post-merge main; the entire delta this branch carries across the regenerated tree is the two rows this card adds. Nothing was clobbered and nothing stale was carried forward.

Post-merge suite

pnpm --filter @objectstack/spec test
Test Files  378 passed (378)
     Tests  9947 passed (9947)

Up from 377 / 9892 pre-merge — the extra file is #7657's view-submit-redirect-url.test.ts, green alongside this change. check-nul-bytes OK over 7117 tracked files.

Handing back for flip + enqueue.

Generated by Claude Code

fix(spec): remove FieldReferenceSchema from the $between endpoints an…

feat(spec): user:profile is explicitly not author-placeable — refused…

chore(console): bump the objectui pin to 00d3f09c5 (152 commits, the …

fix(spec): register the #7596 endpoint transition and declare the nar…

chore(service-analytics): record the where-door face-arm measurement …

fix(spec): refuse a blank $between endpoint at the runtime filter door (

fix(spec): refuse a { $field } reference as a $between endpoint at th…

fix(service-analytics)!: the analytics `where` door runs every arm of…

fix(spec)!: a filter carrying a comparand the query faces refuse is r…

test(spec): the fourth data/ file group's test titles state each cite…

Navigation Menu

Repository navigation

[spec] `FieldReferenceSchema` is declared in the `$between` endpoints but NO backend resolves a `$field` inside a list #7596

The fact

Why it matters

The decision this needs

Refs

OS-DEV-REPORT — #7596

2 remaining items

Merge

os-regen routing (read live from `.gitattributes`, 12 routed paths)

Survival assertions

Post-merge suite

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[spec] FieldReferenceSchema is declared in the $between endpoints but NO backend resolves a $field inside a list #7596

Description

The fact

Why it matters

The decision this needs

Refs

Activity

os-zhuang commented on Aug 11, 2026

os-zhuang commented on Aug 11, 2026

os-zhuang commented on Aug 11, 2026

os-zhuang commented on Aug 11, 2026

os-zhuang commented on Aug 11, 2026

OS-DEV-REPORT — #7596

os-zhuang commented on Aug 11, 2026

2 remaining items

os-zhuang commented on Aug 11, 2026

Merge

os-regen routing (read live from .gitattributes, 12 routed paths)

Survival assertions

Post-merge suite

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[spec] `FieldReferenceSchema` is declared in the `$between` endpoints but NO backend resolves a `$field` inside a list #7596

os-regen routing (read live from `.gitattributes`, 12 routed paths)