Skip to content

fix(service-analytics): the ObjectQL strategy hands the engine its null tests as $null, so a $not over a multi-valued lookup gets the engine's rows (#20918) - #21036

Merged
objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-20918-objectql-not-multivalue
Oct 1, 2026
Merged

objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-20918-objectql-not-multivalue

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #20918
Clause-②: yes

On the engine-aggregate (ObjectQL) strategy, a $not over a multi-valued lookup was refused 400 INVALID_FILTER, while engine.find and the native strategy answered rows. It now gets the engine's rows. The fix is one producer spelling in ObjectQLStrategy.convertFilter. The set / notSet leaves reach the engine as { $null: false } / { $null: true }. They used to arrive as { $ne: null } and the bare null, and driver-sql refuses both over a JSON column. No driver, engine or spec file is touched.

Mechanism assumptions, measured

All readings use one fixture: a ledger with title (text), owner (lookup) and owners (multiple lookup). The rows are a ['u1'], b ['u2'], c ['u2','u1'], d ['u3'], and e, which has owner and owners NULL. The composition is the shipped one: the real SecurityPlugin, ObjectQL and SqlDriver (better-sqlite3), queried as a member. The base is 9b0de7de7, and "after" is this branch.

  • G1, the reference. engine.find for { $not: { owners: { $contains: 'u1' } } } answers b, d, e. The single-valued control { $not: { owner: { $contains: 'u1' } } } also answers b, d, e. engine.aggregate grouped by title answers the same rows for every where row below.

  • G2, where the guard comes from. The analytics where door adds { owners: { $null: false } } beside the $contains inside the $not operand, and it does so twice:

    • the shared lowering, lowerFilterCondition rule 3 (packages/spec/src/data/filter-lowering.ts:445-446, guardNullPolarity's requireValue arm), applied at filter-normalizer.ts:2467;
    • this package's own copy, nullSafeNegationOperand (filter-normalizer.ts:1436, then guardFieldEntry at :1652).

    The guard exists because of SQL's three-valued logic. NOT (owners LIKE '%u1%') is UNKNOWN when owners is NULL, and a WHERE keeps only TRUE, so row e would drop out of a negation that the JS backends answer TRUE for. $not 的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {} 一个是 TRUE 一个是 FALSE #5146 ruled the JS answer canonical. A total leaf, NOT (owners IS NOT NULL AND owners LIKE ...), keeps row e. Both copies of the guard become set leaves. ObjectQLStrategy.convertFilter turned set into { $ne: null } at objectql-strategy.ts:1920 on the base (not :1872), and notSet into the bare null at :1921. notSet carries the allowNull escape of $not operands and of $ne / $nin / $notContains (非否定路径上的 $ne / $nin / $notContains:driver-sql 排除 NULL 行,driver-memory / formula 返回它们(#5146 只裁定了 $not) #5298).

  • G3, the engine already carries the guard. resolveThenLowerWhere (packages/objectql/src/engine.ts:1204) runs the same lowering on where, aggregations[i].filter and having. Measured: engine.find hands driver-sql { $not: { $and: [{ owners: { $null: false } }, { owners: { $contains: 'u1' } }] } } and answers b, d, e. The NULL row e is IN the answer. That is the guard's intent: the negation answers a row with no value instead of dropping it. driver-sql applies $null / $exists on a JSON column (they are absent from JSON_COLUMN_INCOMPATIBLE_OPERATORS). It refuses $ne and the bare equality spelling for any comparand, null included.

  • Which route, from measurement. "Stop adding the guard" would not serve rows 5 to 7 of the table below, { owners: { $null: false } }, { owners: { $exists: true } } and { owners: { $null: true } }. They carry no guard at all: the author's own null test becomes the same set / notSet leaf, so the defect is the leaf's spelling, not the guard. That route would also need the shared normalizer to diverge per strategy, because the native compile and the /analytics/sql echo read the same tree. The other route, "the guard in a form driver-sql applies", serves every row. It changes one producer, and it speaks the lowering's own spelling, so the engine now recognises the guard as one it carries. Measured on the base: the driver received three owners guards for the card's filter, one of them added by the engine. After the fix it receives two, and the engine adds none. (The two are this package's lowering and its hand copy, which is $not 的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {} 一个是 TRUE 一个是 FALSE #5146's documented idempotent double.)

  • G4, the native strategy is unchanged. Every row below answers the same before and after. The native strategy compiles its own SQL, and native-sql-strategy.ts is untouched.

  • G5, Clause-②: yes. Widening: 9 where rows and 3 measure-filter rows go from 400 to rows on the ObjectQL path, at the cube read and at the dataset door. Narrowing: none measured. Every row answered before is answered with the same rows after: every single-valued row, $contains on the multi-valued field, the read scope, and the native strategy on all rows. The 10 re-judged pins below moved only the engine-input spelling. None of their row expectations moved.

Per row

The ObjectQL cell is the same at the cube read (AnalyticsService.query, which POST /api/v1/analytics/query relays) and at the dataset door (POST /api/v1/analytics/dataset/query, selection.runtimeFilter). The native strategy answered the same before and after on every row, so one column carries it.

# filter (where) engine (reference) native, before = after ObjectQL before ObjectQL after
1 { $not: { owners: { $contains: 'u1' } } } b, d, e b, d, e 400 INVALID_FILTER ($ne) b, d, e
2 { $not: { owners: { $notContains: 'u1' } } } a, c a, c 400 (bare equality) a, c
3 { $not: { $or: [{ owners: { $contains: 'u1' } }, { title: 'b' }] } } d, e d, e 400 ($ne) d, e
4 { owners: { $notContains: 'u1' } } b, d, e b, d, e 400 (bare equality) b, d, e
5 { owners: { $null: false } } a, b, c, d a, b, c, d 400 ($ne) a, b, c, d
6 { owners: { $exists: true } } a, b, c, d a, b, c, d 400 ($ne) a, b, c, d
7 { owners: { $null: true } } e e 400 (bare equality) e
8 { owners: { $ne: null } } 400 ($ne) a, b, c, d 400 ($ne) a, b, c, d (see Acceptance notes)
9 { owners: null } 400 (bare equality) e 400 (bare equality) e (see Acceptance notes)
10 { owners: { $contains: 'u1' } } a, c a, c a, c a, c
11 { $not: { owner: { $contains: 'u1' } } } (single-valued control) b, d, e b, d, e b, d, e b, d, e
12 { $not: { owner: { $eq: 'u1' } } } b, d, e b, d, e b, d, e b, d, e
13 { $not: { title: { $contains: 'a' } } } b, c, d, e b, c, d, e b, c, d, e b, c, d, e
14 { owner: { $ne: null } } a, b, c, d a, b, c, d a, b, c, d a, b, c, d
15 { owner: null } e e e e
16 { owner: { $notContains: 'u1' } } b, d, e b, d, e b, d, e b, d, e

Two more positions were measured, as the dataset door's count over all five rows:

position filter engine.aggregate filter engine.find rows native ObjectQL before ObjectQL after
measure filter { $not: { owners: { $contains: 'u1' } } } 5 3 3 400 3
measure filter { owners: { $null: false } } 4 4 4 400 4
measure filter { owners: { $notContains: 'u1' } } 5 3 3 400 3
measure filter single-valued $null: false / null / $not $contains 4 / 1 / 3 same same same same
host read scope $not $contains multi / $null: false multi / owner $ne null b,d,e / a,b,c,d / a,b,c,d same same same

A read scope reaches the engine as written, not through convertFilter, so it was served before and is unchanged.

What changed

  • packages/services/service-analytics/src/strategies/objectql-strategy.ts, convertFilter: set becomes { $null: false } and notSet becomes { $null: true }. The comment records why.
  • .changeset/20918-objectql-not-multivalue-served.md: @objectstack/service-analytics minor, Clause-②: yes.

The notSet arm is fixed beside the set arm. It is the same defect class in the same function, and the card's scope ("a $not over a multi-valued column") includes row 2, which only the notSet arm serves.

Pins, red first (f45d92d43)

  • packages/rest/src/analytics-not-multivalue-null-guard.test.ts: the shipped composition, both strategies, the cube read and the dataset door, against engine.find. It covers rows 1, 2, 4, 5 and 7, plus the single-valued control (row 11), and each row's reference rows are asserted too. Base: 5 red (ObjectQL, the multi-valued rows) and 7 green (native, and the control). Now 12 of 12 green.
  • packages/services/service-analytics/src/__tests__/objectql-null-predicate-json-column.test.ts: the ObjectQL face over a real ObjectQL on SqliteWasmDriver, against engine.find. It covers the same five multi-valued rows and three single-valued controls, plus the engine-input spelling of set / notSet. Base: 6 red and 3 green. Now 9 of 9 green.

Pins re-judged

Ten cases in five existing files recorded the old engine-input spelling, or read it through a test stand-in engine. Each was respelled, and no row expectation moved:

file cases disposition
filter-normalizer-not-null-safe.test.ts 2 engine input respelled to { $null: false } / { $null: true }
filter-value-type-fidelity.test.ts 1 respelled; its strict-equality stand-in engine reads the $null predicate
objectql-contains-canonical-operator.test.ts 2 respelled; its regex-face stand-in reads { $null: flag } where it read the bare null
objectql-daterange.test.ts 1 respelled; its bridge reads $null
where-boolean-flag-refusal.test.ts 4 the control families' engine-input table respelled; their real-engine rows are unchanged

Ablations, predicted before running

The ablations ran at b6c4893fd. Between that commit and the head, b5a2e4481, nothing under packages/services or packages/rest changed. Each ablation:

  • mutated the committed objectql-strategy.ts through scripts/ablation-replace.mjs (anchor 1 to 0, blob 63ce11d10a04 to 75b05cae8989 / aeab45c348ec / 3f893a607ea2);
  • rebuilt @objectstack/service-analytics, and passed ablation-dist-preflight with the old arm present in dist/;
  • ran the unit files and the route pin;
  • restored, proving the blob equal to HEAD 63ce11d10a04, git diff HEAD empty and the tree clean, then rebuilt and passed ablation-dist-preflight --absent for each marker.

Every observed count equals its prediction:

ablation mutation unit (the 6 touched files) route pin (12)
A0 both arms back to { $ne: null } / null 16 red. The full package suite also went 16 red and nothing else (151 files, 3451 passed) 5 red (ObjectQL rows 1, 2, 4, 5, 7)
A1 the set arm only 7 red: new pin 3, not-null-safe 2, boolean-flag 2 ($null: false, $exists: true) 2 red (rows 1, 5)
A2 the notSet arm only 13 red: new pin 4, not-null-safe 1, type-fidelity 1, contains-canonical 2, daterange 1, boolean-flag 4 3 red (rows 2, 4, 7)

Verification, at b5a2e4481 (main 2f2fa11d7 merged, no rebase; it touched nothing on this surface)

  • @objectstack/service-analytics: test exit 0, 151 files, 3467 passed. typecheck exit 0, with 151 of 151 test files in the tsc program (--listFiles).
  • @objectstack/rest: typecheck exit 0 (tsc --noEmit and check:test-typecheck). Its 16 analytics-*.test.ts files: 225 passed and 3 skipped.
  • Consumer sweep, narrowed to the files that load this package: @objectstack/runtime analytics-* plus cross-field-refusal-operand-withhold, 5 files, 38 passed and 4 skipped.
  • Gates: dispatch-gates --commands derived 62. All 62 were run as one locked sequential script, plus the 4 roster families (check-changeset-fixed, check:authz-resolver, check:error-code-casing, check:filter-alias-parity), and each exit code was captured before any pipe.
    • 61 exit 0. check:type-check-debt first exited 3 (prerequisite: @objectstack/runtime and @objectstack/driver-turso unbuilt). It was re-run after building that closure and exited 0: "none above its recorded number".
    • check:dual-build-cjs-loads exited 3 (prerequisite: 38 packages with no dist/, a whole-workspace build). It is NOT MEASURED here.
    • dispatch-gates --ran: 62 derived, 61 run, 1 NOT-MEASURED, 0 UNRUN.
    • The 4 roster families exit 0.
  • Lint, narrowed and proved at b5a2e4481. The population is the 8 changed .ts files, and none is ignored by eslint's own config (isPathIgnored false for all 8). parserOptions.project and projectService are unset for all 8, so no type-aware rule runs and no untouched file's verdict can move. eslint --no-inline-config --format json over them: 8 files, 0 errors, 0 warnings.

Acceptance notes


Generated by Claude Code

claude added 5 commits October 1, 2026 02:07
…ngine's rows on both strategies (red)

Two pins, red on the base for the ObjectQL strategy only:

- packages/rest/src/analytics-not-multivalue-null-guard.test.ts: the shipped
  composition, both strategies, the cube read and the dataset door, against
  engine.find. Base: 5 red (objectql, multi-valued rows), 7 green (native, and
  the single-valued control).
- service-analytics objectql-null-predicate-json-column.test.ts: the ObjectQL
  face over a real engine on SqliteWasmDriver, plus the engine-input spelling of
  the set / notSet leaves. Base: 6 red, 3 single-valued controls green.

Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H
Co-authored-by: Claude <noreply@anthropic.com>
…n spelling, so a $not over a multi-valued lookup is served

ObjectQLStrategy.convertFilter spelled the `set` leaf `{ $ne: null }` and the
`notSet` leaf the bare `null`. Those leaves carry the where door's NULL-safe
guards (the `$not` operand's, and the negative-polarity escape), and
driver-sql refuses both spellings over a multi-valued lookup's JSON column, so
`{ $not: { owners: { $contains: 'u1' } } }` answered 400 INVALID_FILTER where
the engine and the native strategy answer rows. They are now `{ $null: false }`
and `{ $null: true }`, the spellings the engine's own lowering emits for the
same guards.

Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H
Co-authored-by: Claude <noreply@anthropic.com>
…ne spelling of the null predicates

Ten cases in five files pinned the engine input `{ $ne: null }` / the bare
`null`, or evaluated it with a test stand-in that read only those spellings.
Each is respelled, and no row expectation moved:

- engine-input equality respelled to `{ $null: false }` / `{ $null: true }`:
  filter-normalizer-not-null-safe (2), filter-value-type-fidelity (1),
  objectql-contains-canonical-operator (1), objectql-daterange (1),
  where-boolean-flag-refusal (4 control families, real engine rows unchanged);
- the stand-in engines taught the `$null` predicate in place of the bare
  `null`: objectql-contains-canonical-operator's regex face,
  objectql-daterange's bridge, filter-value-type-fidelity's strict-equality
  engine.

Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H
Co-authored-by: Claude <noreply@anthropic.com>
…ved over a multi-valued lookup

Clause-②: yes, measured: refused filters are now answered; no answered filter
changed its rows on the measured matrix.

Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added the size/l label Oct 1, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

2 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 2f2fa11d756f665a4c06160480c1dce15b9d67a4 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from fa95f888a9d0d0eb062c040098783b0e90ad3dc1 — the merge of head b5a2e4481114333fcc08b034aecd28d6a4428af9 into base 2f2fa11d756f665a4c06160480c1dce15b9d67a4, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin fa95f888a9d0d0eb062c040098783b0e90ad3dc1 && git checkout fa95f888a9d0d0eb062c040098783b0e90ad3dc1
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 2f2fa11d756f665a4c06160480c1dce15b9d67a4 b5a2e4481114333fcc08b034aecd28d6a4428af9 && git checkout -B drift-repro 2f2fa11d756f665a4c06160480c1dce15b9d67a4 && git merge --no-ff b5a2e4481114333fcc08b034aecd28d6a4428af9

node scripts/docs-audit/affected-docs.mjs --json 2f2fa11d756f665a4c06160480c1dce15b9d67a4

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: b5a2e4481114333fcc08b034aecd28d6a4428af9
Local-runs: none

Isolated reviewer for PR #21036 (card #20918). Inputs: the card body and its four comments (triage 5918567230, the claim, the dev report 5923712419, the seat ACCEPT), the PR body and file list, the net diff against main at the head (2f2fa11d7..b5a2e4481, 9 files, +535 / -29), and the check-runs on the head. Source cited below was read from the head's tree objects, nothing was built or run.

Check-runs on the head: 34, all completed — 31 success, 3 skipped (Build Docs and Console Pin Gate are path-filtered, Packed-tarball smoke is opt-in), 0 failure, 0 pending. The required contexts (TypeScript Type Check, Test Core, Dogfood Regression Gate, Build Core, Temporal Conformance (live PG + MySQL), Governed Surface Queue Guard) and Lint & Repo Gates, Check Changeset, the four Type Check jobs and the claim guards are all success.

① Derived judgments

  1. Accept set, engine-aggregate (ObjectQL) path — WIDENED, judged right. The only production change is ObjectQLStrategy.convertFilter: the set leaf becomes { $null: false } (was { $ne: null }) and the notSet leaf { $null: true } (was the bare null). driver-sql's assertOperatorAppliesToColumn throws only for an op in JSON_COLUMN_INCOMPATIBLE_OPERATORS ($eq, $ne, the ordering family, $in, $nin, $between); $null and $exists are absent from it. So on a multi-valued lookup's JSON column, every filter the where door lowers to those leaves — a $not operand's guard ($not 的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {} 一个是 TRUE 一个是 FALSE #5146), the $notContains escape (非否定路径上的 $ne / $nin / $notContains:driver-sql 排除 NULL 行,driver-memory / formula 返回它们(#5146 只裁定了 $not) #5298), an author's own $null / $exists — goes from 400 INVALID_FILTER to rows, at the cube read, the dataset door's selection.runtimeFilter, and a dataset measure's filter. That is a widening of what the door accepts on one strategy, and the declaration says yes. Right.
  2. "The engine's own spelling" — right. lowerFilterCondition rule 3 (guardNullPolarity, packages/spec/src/data/filter-lowering.ts) emits { f: { $null: false } } for the requireValue arm and { $or: [{ f: { $null: true } }, { f: spec }] } for allowNull; the engine runs it on every driver at resolveThenLowerWhere (engine.ts:1204). operatorIsNullTotal('$null') is true, so the engine reads the strategy's guard as already total and adds none beside it. The dev's "three guards, then two" reading is consistent with source; it was not re-measured here.
  3. Previously-answered queries — spelling-only, no narrowing, judged right. On single-valued columns driver-sql compiles $ne with a null comparand as whereNotNull(field) and $null: false as whereNotNull(field); $eq null / the bare null compile IS NULL and $null: true as whereNull(field) — one predicate either way. The ten re-judged cases in five pins (filter-normalizer-not-null-safe, filter-value-type-fidelity, objectql-contains-canonical-operator, objectql-daterange, where-boolean-flag-refusal): read in the diff, every change is to an engine: expectation or to a stand-in engine's reader of the new shape; every rows: table and every id expectation is unchanged. The native strategy (native-sql-strategy.ts), the /analytics/sql echo and the read scope (reaches the engine without convertFilter) are untouched by the diff.
  4. Rows 8-9 ({ owners: { $ne: null } } and the bare { owners: null }) — a widening already covered by yes, declared, acceptable; not a divergence that blocks. At the analytics door these are null predicates by contract (分析查询的 {field: {$eq: null}} / {$ne: null} 编译成 col = '' / col != '',与同文件里 {field: null} 的 IS NULL 自相矛盾 #5332: fieldLeaves reads $eq / $ne with a null comparand as notSet / set, and raw === null as notSet). The native strategy answered them before; the ObjectQL strategy now answers the same rows, and those rows are exactly what the engine answers for the equivalent $null spelling (rows 5 and 7: a, b, c, d and e). engine.find refuses the $ne / bare spelling only because the driver-sql: a declared operator on a multiple: true (JSON array) column silently answers wrong — $in/$eq always zero rows, $nin returns the rows it was asked to EXCLUDE #7398 gate reads the operator, not the comparand — a fail-closed refusal, not a different answer. The changeset body states the difference ("One difference from the engine remains"), no driver file is touched, and triage's ⛔ on a driver special case is honoured. No new card owed.
  5. Public surface — unchanged, right. No export added or removed; packages/spec, packages/objectql, packages/drivers untouched (the 9 files are the changeset, one packages/rest/src/*.test.ts, and packages/services/service-analytics/src/**). No REST route, query-parameter set, error code or schema moved.
  6. Triage's direction — honoured. Route one of the either/or ("the guard in a form driver-sql applies") with the measured reason that rows 5-7 carry no guard and "stop adding the guard" would leave them refused; filter-normalizer.ts untouched. Pins: the rest pin runs both compositions (native and engine-aggregate) over the shipped SecurityPlugin + ObjectQL + SqlDriver chain against engine.find as the reference, with the single-valued control; the unit pin adds the engine-input spelling assertion. As triage asked.
  7. Non-SQL drivers — reasoning, not measurement, recorded as such. The engine's lowering already hands every driver $null guards on every $not; driver-mongodb's $null arm translates to $eq: null / $ne: null, and driver-memory / driver-turso carry a $null arm. Consistent; not a measured row.

② Semver level

  • .changeset/20918-objectql-not-multivalue-served.md: @objectstack/service-analytics minor, body carries Clause-②: yes. PR body line 2: Clause-②: yes, no arm. Check Changeset success on the head.
  • Grammar — right. yes with no arm declares a widening (yes and yes (widening) are one reading). A (narrowing) arm is not owed: no previously-answered row moved (① item 3). No (widening) arm is required.
  • Level — right. yes takes at least minor; one published package grew its accept set; nothing breaks, so no ADR-0087 marker is owed and none is written.
  • Prose — accurate against the diff. It names the two doors, the strategy, the filter shapes that went from 400 to rows, what is unchanged, and the one remaining difference from the engine.

③ Boundary flags

Implemented-by: claude/issue-20918-objectql-not-multivalue
Reviewed-by: session_01XY5uCwTjZj7884yYtyur4H

VERDICT: PASS

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 1, 2026 03:18
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 1, 2026 03:18
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 1, 2026
Merged via the queue into main with commit 5dbeb7d Oct 1, 2026
36 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20918-objectql-not-multivalue branch October 1, 2026 04:31
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
…n reads the object its lookup field declares (objectstack-ai#20986) (objectstack-ai#21088)

Fixes objectstack-ai#20986
Clause-②: yes (narrowing)

## What this changes

A relationship-path hop the cube declares no join for now reads the
object its lookup field DECLARES as its target, the field's `reference`.
Before, such a hop fell back to its ALIAS, the lookup field's own name.
An inferred cube declares no join, so a dotted path through a lookup
named differently from its target (`owner` referencing a person object)
named no object at all. The door admitted, and refused, the field's name
as if it were an object, and the strategies joined and read a table of
that name.

One resolver, `service-analytics/src/hop-object.ts` (`resolvePathHops`),
names a hop's object in three tiers, first answer wins:

1. the cube's declared join at that path, keyed by the path with its
dots as `__`. An authored cube that declares its join keeps it;
2. the relationship field's declared `reference`, asked of the host's
`relationshipResolver`. `AnalyticsServicePlugin` already wires it from
the data engine's object schema, through `referenceCarrierOf`, for
dataset compilation;
3. the alias itself, when the host cannot answer.

Every reader takes its answer from there, with the same function: the
door's field gate and its admitted and scoped set (`fieldsOfColumnSql`,
`queryObjects`), and both strategies through the context's new
`relationshipReference`, which the service sets to the very function its
own gate uses. There is no second resolution.

- `NativeSQLStrategy`: each registered join records the object it reads
(`StatementJoins`), and the read scope applied to the alias is that
recorded object's. The join table and the scoped object are one value.
- `ObjectQLStrategy`: the cross-object plan and the FK-expand read (and
its read scope) take the hop's object from the resolver. A lookup whose
declared target is the base object itself, a self-reference such as
`parent`, still plans as a cross-object hop.

## Per site (H2), file:line on base `8f784959cf` and on head
`e75208968e`

| site | base | role | head |
|---|---|---|---|
| `analytics-service.ts` `fieldsOfColumnSql` (via `namedQueryFields`) |
`:428` `joins?.[alias]?.name`, else alias | the field gate, AND the
admitted and scoped set (`queryObjects` `:1612`, `assertFieldsReadable`
`:1682`) | `:432` `resolvePathHops`; callers `:1652`, `:1722` pass
`this.hopReference` |
| `analytics-service.ts` `cubeObjects` | `:1634` `j?.name ?? alias` |
admits and scopes the DECLARED joins only | unchanged `:1674`; `??
alias` is unreachable, `CubeJoin.name` is required by the spec |
| `native-sql-strategy.ts` `crossFieldComparisonIn` fallback | `:364`
`cube.joins?.[alias]?.name ?? alias` | reads scopes for the decline,
declared joins only, only for a context without the door's set |
unchanged `:388`; the door always passes its set |
| `native-sql-strategy.ts` `generateSql` read-scope loop | `:686`
`cube.joins?.[alias]?.name ?? alias` | scopes each joined alias | `:751`
`join.object`, the object the join recorded |
| `native-sql-strategy.ts` `qualifyAndRegisterJoin` | `:848`
`cube?.joins?.[alias]?.name ?? alias` | joins | `:904` `resolvePathHops`
|
| `native-sql-strategy.ts` `resolveStorageTarget` | `:1044`
`cube.joins?.[joinAlias(relPath)]?.name ?? relPath` | reads (declared
type, temporal coercion) | `:1104` `columnObjectOf` |
| `native-sql-strategy.ts` `canHandle` external decline | `:169`
declared join targets | reads (routing), declared joins only | unchanged
`:193`; see Acceptance notes |
| `objectql-strategy.ts` `isCrossObjectField` | `:724`
`cube.joins?.[alias]?.name ?? alias` | reads (plans the FK-expand or the
refusal) | `:736` `resolvePathHops` |
| `objectql-strategy.ts` `planCrossObject` cross dimension | `:1054`
`refObject: cube.joins?.[alias]?.name ?? alias` | reads and scopes (the
FK-expand `executeAggregate` and its `getReadScope`) | `:1070`
`resolvePathHops` |
| `objectql-strategy.ts` `resolveStorageTarget` | `:1456` `... ??
relPath` | reads (declared type for the lowering and the echo) | `:1478`
`columnObjectOf` |
| `structured-json-dimension-door.ts` `columnOf` | `:192` declared join
only, stands down otherwise | judges a grouped column's class |
unchanged; outside this claim, see Acceptance notes |
| `dataset-compiler.ts` | `:648` `relationshipResolver` per `include` |
joins (compiled, declared) | unchanged: already tier 2's source, read
into tier 1 |

**H3, where the declared reference is reachable.** At the door,
`AnalyticsServiceConfig.relationshipResolver`. `plugin.ts` answers it
from `engine.getObject(base).fields[rel]` for a `lookup` or
`master_detail` field, through `referenceCarrierOf`. At the strategies
it was not reachable on base: `StrategyContext` carried a field's type
and value shape (`declaredFieldType`, `declaredValueShape`), never its
reference. It is now, as
`DatasetScopedStrategyContext.relationshipReference`, which is
package-internal. The context type is not exported. The member's one hit
in the built `dist/index.d.ts` is prose in the doc comment of the
private `hopReference` field, against 19 hits for `StrategyContext`. A
host returning a `RelationshipTarget` is read by its `object`.

**H4, the divergence check, both before and after.** On base, every site
resolved the alias (the reads site resolved a dotted `relPath`), so the
object admitted and the object joined were the same name. After, the six
changed sites call `resolvePathHops` with the same function. The native
join and the scope applied to it are one recorded value. The unchanged
sites read declared joins only (tier 1), which the resolver returns
verbatim. No site uses the alias where another uses the reference, so
there is no `security` stop. Measured on the route pin: the security spy
shows `canReadObject` asked only for the base object and the target.

## Per row (H1), measured in the shipped composition

Fixture: real `SecurityPlugin` and `AnalyticsServicePlugin` over
`ObjectQL` on SQLite. A member who may read everything except two
objects. The "before" column was read at base `8f784959cf`. The decoy
row's "before" was read on the ablation leg below, whose resolver is the
base one. The "after" column was read at the fix commit `bf58903995`.
The member is the caller. At head `e75208968e`, the two pins re-ran
green over these rows: the readable dimension and filter, the unreadable
target, the self-reference, and the controls.

| strategy | lookup | target | position | before | after |
|---|---|---|---|---|---|
| native | named differently (`owner`) | readable | dimension | 403
naming `owner` (system caller: 500 `DATABASE_ERROR`) | rows, equal to a
declared join's |
| native | named differently | readable | filter member | 403 naming
`owner` (system: 500) | rows, equal to a declared join's and to the
nested form `{ owner: { region } }` |
| native | named differently (`keeper`) | unreadable | dimension, filter
| 403 naming `keeper` | 403 naming the target |
| native | named after its target (control) | readable / unreadable |
dimension | rows / 403 naming it | unchanged |
| native | self-reference (`parent`) | the base object | dimension,
filter | 403 naming `parent` | rows |
| native | named after ANOTHER object (decoy) | unreadable | dimension |
answered from the other object's table | 403 naming the target |
| ObjectQL | named differently | readable | dimension | 403 naming
`owner` (system: 500) | rows (FK-expand), equal to a declared join's |
| ObjectQL | named differently | readable | filter member | 403 naming
`owner` (system: 400) | 400 `INVALID_FIELD`, the strategy's own
capability refusal, equal to a declared join's |
| ObjectQL | named differently | unreadable | dimension, filter | 403
naming `keeper` | 403 naming the target |
| ObjectQL | named after its target (control) | readable / unreadable |
dimension | rows / 403 naming it | unchanged |
| ObjectQL | self-reference | the base object | dimension | 403 naming
`parent` | rows (FK-expand) |
| ObjectQL | named after another object (decoy) | unreadable | dimension
| answered from the other object | 403 naming the target |

**`Clause-②: yes (narrowing)`, measured.** Widening: the first rows move
from refused to served. Narrowing: the decoy rows move from answered to
`403`. A lookup whose name is also the name of ANOTHER object used to
read that other object's rows, by the ids of the records the field
points to. The dispatch expected a plain `yes`; the measured grammar
adds the arm. Line 2 reads `declared · yes · narrowing` through
`scripts/pm/clause2-line.mjs`'s `readClause2Line`. The changeset is
`minor`, carries the BREAKING banner, and carries its ADR-0087 marker
(`not-required (no-migration-prescription)`).

## Pins, red first, and the ablation

- **Unit pin**
`service-analytics/src/__tests__/hop-object-reference-resolution.test.ts`
(28 cases), on both strategies. It covers five positions: an inferred
dimension, a filter member, a time-dimension window, an authored member
over an undeclared relationship, and a two-hop path's second hop. For
each, an unreadable target is refused by name before anything runs, on
the read and on the echo. It also pins:
- admission and the read scope ask one set, carrying the target and
never the field's name;
  - the field gate judges the column on the target;
  - the native join and its scope name the target;
  - the native strategy asks the target's declared column type;
  - the ObjectQL FK-expand reads the target under its scope;
  - the self-reference on both strategies;
- and two controls: a lookup named after its target, and a declared join
is kept.
- **Route pin**
`packages/rest/src/analytics-hop-object-reference.test.ts` (10 cases),
once per strategy. Every answer is compared with the same question
through a DECLARED join and checked absolutely. It covers:
- readable dimension: rows, with only the base and the target admitted
(security spy);
- readable filter: rows on native, equal to the nested form; ObjectQL's
own 400;
- unreadable dimension and filter: 403 naming the target, on the read
and the echo;
  - the control.
- **Red first.** Pins committed at `f31ca819e2`, before the fix at
`bf58903995`: unit 24 failed / 4 passed (the 4 are the controls), route
8 failed / 2 passed (the controls). Every failure sits at the assertion
naming the target, never at a reference assertion.
- **Ablation A1, the resolver falls back to the alias again.** Predicted
in writing before the run: exactly the red set above.
- `scripts/ablation-replace.mjs` replaced tier 2's `const reference =
referenceOf?.(from, field);` with `const reference = undefined as string
| undefined;`: anchor 1 to 0, blob `489bc64f35f0` to `a783c0f62a1c`. The
package was rebuilt, and `ablation-dist-preflight --absent` found the
marker gone from all 6 built files. The pristine build carried it in
`dist/index.js` and `dist/index.cjs`, 1 each.
- Result: unit 24 failed / 4 passed, route 8 failed / 2 passed. The
failing names are identical, as sorted lists, to the pins-first red set.
- Restore: by absolute path, under an EXIT/INT/TERM trap. The blob
equals the HEAD blob (`489bc64f35f0`), `git diff HEAD` is empty, and
porcelain is 0. After a rebuild, the preflight found the marker present
in 2 built files, and the tree clean. The pins then read unit 28/28,
route 10/10.

## Tests and gates, head `e75208968e`

Each command below ran under `os-verify-lock`, in ONE locked sequential
script, on head `e75208968e`. The script printed `git rev-parse --short
HEAD` at its start and its end, with a clean porcelain both times. Each
exit code was captured before any pipe.

- **Build:** `turbo run build --filter=!@objectstack/docs
--concurrency=1`, 72 of 72 tasks, exit 0.
- **Gate union:** `dispatch-gates --commands --repo
objectstack-ai/objectstack` derives 62 commands from this diff's 8
paths, the same 62 as on the first merge. All 62 exit 0.
- `dispatch-gates --ran`: `62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN`.
- Verdict lines include `check:nul-bytes` (OK, 9671 files, no raw
control bytes), `check:cross-package-test-inputs` ("29 package(s) read
outside themselves, all declared"), `check:engine-double-contract`,
`check:dual-build-cjs-loads`, `check:type-check-debt` (no entry above
its count), `check-adr-0087-registration` (the
`no-migration-prescription` exemption read from this changeset) and
`check-empty-changeset`.
- `check-changeset-no-major`'s clause-② axis reads the PR payload, so it
is NOT APPLICABLE locally and is CI's.
- **The four ⛔ roster families the derivation names, run anyway:**
`check-changeset-fixed`, `check:authz-resolver`,
`check:error-code-casing` and `check:filter-alias-parity`. All exit 0.
- **`@objectstack/service-analytics`:** `vitest run --maxWorkers=2`: 155
files, 3526 passed, 10 skipped. `typecheck` exits 0, and `--listFiles`
counts 152 of the 152 `__tests__` files, the unit pin among them, in the
program.
- **The rest analytics pins:** every `src/analytics-*.test.ts` plus
`data-nested-relation-permission.test.ts`, 20 files: 243 passed, 8
skipped. `@objectstack/rest` `typecheck` exits 0, with
`check:test-typecheck` OK; the route pin is in `tsconfig.test.json`'s
program (`--listFiles`).
- **The client census** `envelope-caller-census.test.ts`: 20 of 20.
Neither pin calls the censused spelling: the pins call `service.query(`,
never `analytics.query(`.
- **ESLint, a proven narrowing.**
- The population is the 7 changed `.ts` files (`git diff --name-only
2821e9f...HEAD`). Per eslint's own config, `isPathIgnored` is false
for all 7, and `parserOptions.project` and `projectService` are unset
for all 7.
  - `lintFiles` gives 7 results, 0 errors, 0 warnings.
- With no type-aware lint, this diff cannot move an untouched file's
verdict.
- **On the first merge (`cff14ac80f`)**, the same union was 62 of 62
exit 0, and the four rosters were green. Analytics was 154 files / 3521
passed, and the rest pins 19 files / 242 passed.

## Surface

Within the claim's regions: `fieldsOfColumnSql` and `queryObjects` in
`analytics-service.ts`; the hop-object sites in both strategies; the new
module; the pins; and the changeset. ⛔ Not touched: the native
`execute`, `buildFilterClause`, `convertFilter`,
`packages/objectql/src/**` or `packages/spec/src/**`.

Four further lines carry the resolver to the strategies, outside the
claim's listed regions:

- `analytics-service.ts`: the `hopReference` class field,
`baseCtx.relationshipReference`, the `assertFieldsReadable` call to
`namedQueryFields`, and the `relationshipResolver` config doc;
- `strategies/types.ts`: one member, `relationshipReference`, on the
package-internal context type.

No in-flight sibling PR touches them. `main` was merged twice, with no
rebase: `5dbeb7d7b7` brought PR objectstack-ai#21036 (`convertFilter`), and
`2821e9f15b` brought PR objectstack-ai#21040 (`execute`) and PR objectstack-ai#21037. Both merges
were clean, and the branch delta against `main` stayed the same 8 files.

## Acceptance notes

- **The ObjectQL strategy still refuses a dotted FILTER or time window
through any relationship** with its own `400 INVALID_FIELD`, as it does
through a declared join. The engine serves the nested form there.
Lowering the dotted filter onto the nested form for the engine belongs
to `convertFilter`, outside this claim. The dimension position is served
(FK-expand).
- **The grouped-column door** (`structured-json-dimension-door.ts`
`columnOf`) judges declared joins only and stands down on a path the
cube does not declare. A multi-value or structured-JSON column reached
through an undeclared path is not judged there. That was already true
for a lookup named after its target, and is now reachable for one named
differently. NOT MEASURED. Outside this claim.
- **The native external-object decline** (`canHandle`) reads declared
joins only, so an undeclared hop to a federated object is not declined.
That holds for same-named lookups on base, and now for differently named
ones. NOT MEASURED: there is no federated fixture.
- **The ObjectQL SQL echo for a self-reference** prints an unaliased
self-join. It describes a statement that is not the FK-expand the query
runs. NOT MEASURED by a pin; it affects the echo only.
- **A null FK** buckets as `(restricted)` on the ObjectQL FK-expand and
as null on native. This is pre-existing, and the same for a lookup named
after its target.
- **A hop no tier answers** (a field that declares no reference, or a
host with no resolver) reads the alias, as before. For a multi-hop path
that fallback is now spelled as the join alias (`a__b`) on the two reads
sites, where it was the dotted path. Either spelling names no object.
- **`relationshipResolver`'s warn** for a field whose `reference` is not
a string is now reachable per query hop, not only at dataset
compilation. NOT MEASURED.
- **The dataset door** with a selection naming an undeclared,
differently named path is NOT MEASURED. The route pin covers the cube
read and the echo.
- **Drivers.** SQLite only. The resolution is computed before any
driver.
- **CI is not awaited.** These lanes are CI's: the test shards, dogfood,
temporal conformance, build core, the workspace type-check lanes, and
the wide-population and roster families the derivation lists outside its
total.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants