fix(service-analytics)!: the analytics read scope and the native where answer $contains on a JSON-stored field by membership (#20987)

.changeset/20987-analytics-contains-membership.md

@@ -0,0 +1,21 @@
+---
+'@objectstack/core': minor
+'@objectstack/service-analytics': minor
+'@objectstack/driver-sql': patch
+---
+
+fix(service-analytics)!: the analytics read scope and the native `where` answer `$contains` / `$notContains` on a multi-valued or JSON-stored field by membership, with the one construct `driver-sql` emits, now exported from `@objectstack/core` (#20987)
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a correction of which rows two analytics SQL faces answer for one operator on one declared field class: the read scope `compileScopedFilterToSql` compiles from a row policy, and the native strategy's rendering of a query's `where`. No authorable key, spelling, export or stored shape of metadata moves; `packages/spec` is untouched, and the contract sentence the faces now meet (`FILTER_OPERATORS.$contains`) is the one already declared. A policy or filter that was written stays written as it was, and what it now selects is what the data door already selected for it, so there is nothing a ledger entry could rewrite. The new refusal on a datasource whose SQL dialect the host cannot name is a refusal of a query, not of stored metadata. The other categories are closed on facts: the packages publish (not `unpublished`); no ADR-0087 id covers a filter operator's reading (not `registered` / `already-registered`); and the change is runtime behaviour plus one additive export, not a declaration change (not `runtime-interface-only` / `type-surface-only`). -->
+
+**BREAKING**: this narrows what the analytics doors answer for one class of read. A row policy (the read scope the analytics plugin compiles from the security service, or a host's own `getReadScope`) whose `$contains` or `$notContains` names a field declared multi-valued (`multiple: true` on a multi-capable type, or a multi-option type) or JSON-stored now selects the rows holding the comparand as an ELEMENT of the stored list. It used to select every row whose stored JSON text contained the comparand as a substring, so on SQLite a policy could admit rows outside it, and on PostgreSQL every query under such a policy answered `500` (MySQL was not measured). An analytics count under such a policy now equals what the same caller reads through the data door. On a datasource whose SQL dialect the analytics host cannot name, such a policy now refuses the query (`READ_SCOPE_COMPILE_FAILED` / `500`) instead of falling back to the substring reading. It ships as `minor` under the launch-window convention.
+
+**The `where`.** `POST /api/v1/analytics/query`, the dataset door and `/analytics/sql` on the native strategy render the same membership test for a `$contains` / `$notContains` in a query's `where` (or a dataset's `runtimeFilter`) on such a field: on PostgreSQL the query answers rows where it answered `500`, and on SQLite the count stops over-counting (`$contains`) and under-counting (`$notContains`). On a datasource whose dialect the host cannot name, the operator on such a field is refused `INVALID_FILTER` / `400`. The ObjectQL strategy already answered membership and is unchanged.
+
+**Unchanged.** On a scalar text field `$contains` stays the substring test, on every face. `$notContains` keeps its NULL rule: a row with no value satisfies it. A host that wires no field metadata keeps the substring reading, because it cannot tell a JSON column from a text one; the analytics plugin wires it from the data engine.
+
+**New export.** `@objectstack/core` exports `jsonMembershipPredicate(dialect, emitters, value)` and `jsonMembershipCandidates(value)`, with the `JsonMembershipDialect` and `JsonMembershipEmitters` types: the per-dialect membership construct (#17590) moved from `@objectstack/driver-sql`, where it was module-private, and made placeholder-agnostic. `@objectstack/driver-sql` imports it and emits byte-identical statements and bindings.
+
+**What to do after upgrading.** Nothing, unless a policy or a dashboard filter relied on the substring reading of a multi-valued or JSON-stored field: such a filter now selects members only, as the data door always did. A host whose analytics `sqlDialect` hook answers nothing for a SQL datasource should answer `'sqlite'`, `'postgres'` or `'mysql'`, or the operator on such a field is refused.

packages/core/src/index.ts

@@ -56,6 +56,7 @@ export * from './utils/datetime.js';
 // runtime dependencies, and a copy per face is how one `sum` came to answer
 // two doubles.
 export * from './utils/compensated-sum.js';
+export * from './utils/json-membership-sql.js';
 // [#20889] What an aggregate ANSWERS and its `'number'` presenter, moved from
 // `driver-sql` so the analytics native-SQL face presents with the same rule.
 export * from './utils/aggregate-answer.js';

packages/core/src/utils/json-membership-sql.test.ts

@@ -0,0 +1,111 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#20987] `jsonMembershipCandidates` and `jsonMembershipPredicate` — the one
+ * `$contains` membership construct `driver-sql` and the analytics read scope
+ * and `where` ask, moved here from `driver-sql` (#17590).
+ *
+ * What each face does with the SQL is pinned in its own package (the driver's
+ * move proof `sql-driver-20987-json-membership-move.test.ts`, its executed
+ * three-dialect suite `sql-driver-17590-json-column-membership.test.ts`, and
+ * the analytics `contains-membership.test.ts`). This file pins the function's
+ * own contract: the candidate set, the construct per dialect, and that the
+ * emitters are called left to right, in placeholder order, or not at all.
+ */
+
+import { describe, it, expect } from 'vitest';
+import { jsonMembershipCandidates, jsonMembershipPredicate, type JsonMembershipEmitters } from './json-membership-sql';
+
+/** Emitters that render `C` for the column and `?` for a value, recording each call in order. */
+function recording(): { emit: JsonMembershipEmitters; calls: string[] } {
+  const calls: string[] = [];
+  return {
+    calls,
+    emit: {
+      column: () => {
+        calls.push('column');
+        return 'C';
+      },
+      value: (v) => {
+        calls.push(`value ${v}`);
+        return '?';
+      },
+    },
+  };
+}
+
+describe('[#20987] jsonMembershipCandidates — the JSON scalars one comparand denotes', () => {
+  it('a plain string denotes one JSON string', () => {
+    expect(jsonMembershipCandidates('u1')).toEqual(['"u1"']);
+  });
+
+  it('a string JSON must escape is escaped', () => {
+    expect(jsonMembershipCandidates('say "hi"')).toEqual(['"say \\"hi\\""']);
+  });
+
+  it('true / false / null also denote their JSON literal', () => {
+    expect(jsonMembershipCandidates('true')).toEqual(['"true"', 'true']);
+    expect(jsonMembershipCandidates('false')).toEqual(['"false"', 'false']);
+    expect(jsonMembershipCandidates('null')).toEqual(['"null"', 'null']);
+  });
+
+  it('a JSON number also denotes the number, canonicalised', () => {
+    expect(jsonMembershipCandidates('7')).toEqual(['"7"', '7']);
+    expect(jsonMembershipCandidates('1.50')).toEqual(['"1.50"', '1.5']);
+    expect(jsonMembershipCandidates(10)).toEqual(['"10"', '10']);
+  });
+
+  it('a string Number() accepts but the JSON number grammar refuses denotes the string only', () => {
+    for (const text of ['0x10', ' 1 ', 'Infinity', '', '01', '1.']) {
+      expect(jsonMembershipCandidates(text), JSON.stringify(text)).toEqual([JSON.stringify(text)]);
+    }
+  });
+});
+
+describe('[#20987] jsonMembershipPredicate — the construct per dialect, through the caller\'s emitters', () => {
+  it('sqlite: a json_each element scan, the column twice then the value, per candidate', () => {
+    const { emit, calls } = recording();
+    expect(jsonMembershipPredicate('sqlite', emit, 'u1')).toBe(
+      "(EXISTS (SELECT 1 FROM json_each(CASE WHEN json_valid(C) THEN C ELSE '[]' END) AS os_member "
+        + "WHERE typeof(os_member.key) = 'integer' AND CASE os_member.type "
+        + "WHEN 'true' THEN 'true' WHEN 'false' THEN 'false' WHEN 'null' THEN 'null' "
+        + 'ELSE json_quote(os_member.value) END = ?))',
+    );
+    expect(calls).toEqual(['column', 'column', 'value "u1"']);
+  });
+
+  it('postgres: jsonb containment of the array-wrapped candidate', () => {
+    const { emit, calls } = recording();
+    expect(jsonMembershipPredicate('postgres', emit, 'u1')).toBe('(C::jsonb @> ?::jsonb)');
+    expect(calls).toEqual(['column', 'value ["u1"]']);
+  });
+
+  it('mysql: JSON_CONTAINS of the array-wrapped candidate', () => {
+    const { emit, calls } = recording();
+    expect(jsonMembershipPredicate('mysql', emit, 'u1')).toBe('(JSON_CONTAINS(C, ?))');
+    expect(calls).toEqual(['column', 'value ["u1"]']);
+  });
+
+  it('two candidates are OR-ed inside one pair of parentheses, emitted left to right', () => {
+    const { emit, calls } = recording();
+    expect(jsonMembershipPredicate('postgres', emit, '1.50')).toBe('(C::jsonb @> ?::jsonb OR C::jsonb @> ?::jsonb)');
+    expect(calls).toEqual(['column', 'value ["1.50"]', 'column', 'value [1.5]']);
+  });
+
+  it("'unknown' answers null and calls neither emitter", () => {
+    const { emit, calls } = recording();
+    expect(jsonMembershipPredicate('unknown', emit, 'true')).toBeNull();
+    expect(calls).toEqual([]);
+  });
+
+  it('a positional caller binds in placeholder order', () => {
+    const params: unknown[] = [];
+    const sql = jsonMembershipPredicate(
+      'postgres',
+      { column: () => '"t"."tags"', value: (v) => { params.push(v); return `$${params.length}`; } },
+      'true',
+    );
+    expect(sql).toBe('("t"."tags"::jsonb @> $1::jsonb OR "t"."tags"::jsonb @> $2::jsonb)');
+    expect(params).toEqual(['["true"]', '[true]']);
+  });
+});