fix(runtime,objectql): ctx.title() answers a title formula over lookups — the one hop is served on the body's own channel

.changeset/24197-objectql-serve-formula-field.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/objectql': minor
+---
+
+feat(objectql): evaluate a formula field over a record in hand the way a read serves it, one hop through a reference included
+
+A formula title over lookups (`record.book.title + ' · ' + record.reader.name`) is computed on every read since the formula read path reads one hop. A record no read returned, such as the state a hook fires on, had no way to get the same value: the synchronous `evaluateFormulaField` reads no hop, so it answered `null`.
+
+- **`ScopedContext.serveFormulaField(objectName, record, field)`** evaluates one declared formula field over `record` with the read path's own evaluation, under that context: the one hop it reads through a reference (one batched read per reference field, through the reader's own permissions, on the context's transaction when it carries one), the `can()` permission map, and the fault report. A related record the reader may not read, a missing one and a second hop leave the field `null` with the read's reason, as a read does. `ObjectQL.serveFormulaField(objectName, record, field, context)` is the engine method behind it.
+- **`resolveServedRecordTitle(schema, record, channel)`** is the async sibling of `resolveRecordTitle`: this record's title, with a formula title served on `channel` (a `ScopedContext`). `resolveRecordTitle` is unchanged and still reads no hop.
+- **`servedRecordTitle(schema, row)`** reads the title a read already served off the row it returned, instead of re-deriving it.
+
+What to do. Nothing. Code that names a record it holds in hand and wants the title a read shows calls `resolveServedRecordTitle` with the caller's `ScopedContext`.

.changeset/24197-runtime-hook-title-one-hop.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/runtime': patch
+---
+
+fix(runtime): `ctx.title()` in a hook body answers a title formula that reads through a lookup, as a read of the record shows it
+
+A junction object's composed title, `record.book.title + ' · ' + record.reader.name`, is shown by every read of the record, but a hook body's `ctx.title()` answered `null` for it, and `ctx.title('<lookup field>')` answered `null` for a related record whose own title reads through a lookup.
+
+- **`ctx.title()`** now reads the one hop the title formula reads, through the body's own `ctx.api` channel: one read per lookup the title reads through, with the caller's permissions, on the transaction the body has open. It follows the lookup's value in this write, so a `beforeInsert` names the payload's book and a `beforeUpdate` that changes `book` names the new book. A title over the record's own columns still costs no read.
+- **`ctx.title('<lookup field>')`** returns the title its `findOne` served, the related record's own one-hop title included, instead of re-computing it without the hop.
+- A related record the caller may not read leaves the title `null`, as the read of the same record shows no title to that caller. It is never a partial title and never the hidden name. A title that would need a second hop is `null`, as it is on a read.
+
+What to do. Nothing. A hook that composed a junction object's title by hand can call `ctx.title()` instead.

content/docs/automation/hook-bodies.mdx

@@ -100,8 +100,8 @@ The script sees only what the surrounding `ctx` object exposes:
 | `ctx.user` / `ctx.session` | Identity context. | none |
 | `ctx.api.object(name).find\|count\|aggregate` | Cross-object reads, scoped to current tenant. | `api.read` |
 | `ctx.api.object(name).insert\|update\|delete` | Cross-object writes. | `api.write` |
-| `ctx.title()` | This record's title — the object's `nameField`, **including when it is a `formula`** (evaluated server-side against the record already in hand, no extra read). | none |
-| `ctx.title('<lookup field>')` | The related record's title, through a lookup / master_detail / user / tree column. Costs one `findOne`. | `api.read` |
+| `ctx.title()` | This record's title — the object's `nameField`, **including when it is a `formula`** (evaluated server-side against the record already in hand: no extra read over the record's own columns, one read per lookup a title such as `record.book.title + ' · ' + record.reader.name` reads through). | none |
+| `ctx.title('<lookup field>')` | The related record's title, through a lookup / master_detail / user / tree column. Costs one `findOne` (plus the lookups that record's own title reads through). | `api.read` |
 | `ctx.crypto.randomUUID()` | UUID generation. | `crypto.uuid` |
 | `ctx.log.{info,warn,error}` | Structured logging. | `log` |
 | `ctx.connector(name).<method>(...)` _(planned)_ | Outbound HTTP / SaaS calls. **Not yet wired into the sandbox** — ships with the separate Connector spec. | (separate Connector spec) |
@@ -127,14 +127,21 @@ const account = await ctx.title('account_id');
 
 Three properties worth knowing:
 
-- **A formula `nameField` costs nothing extra.** It is evaluated server-side
-  against the record the hook is already firing on — the same expression, the
-  same evaluator and the same rounding a `GET` of that record would use, so the
-  title a hook writes and the title the UI shows cannot drift.
-- **The related form costs exactly one `findOne`**, through your body's own read
+- **A formula `nameField` is evaluated the way a `GET` evaluates it.** Against
+  the record the hook is already firing on — the same expression, the same
+  evaluator and the same rounding a `GET` of that record would use, so the title
+  a hook writes and the title the UI shows cannot drift. A title over the
+  record's own columns costs nothing extra. A title that reads one hop through a
+  lookup (`record.book.title + ' · ' + record.reader.name`) costs one read per
+  lookup it reads through, follows the lookup's value in this write (a
+  `beforeUpdate` that changes `book` names the new book), and is read through
+  your body's own read channel: a related record you may not read leaves the
+  title `null`, exactly as the `GET` shows no title.
+- **The related form costs one `findOne`**, through your body's own read
   channel — so it obeys the caller's scope and joins an open
   `ctx.api.transaction`. That is why it requires `api.read` while the bare form
-  requires nothing: the token gates the read, and there is no read to gate.
+  requires nothing: the token gates your body's reach for another record; the
+  bare form names this record by its own declaration.
 - **It never falls back to the id.** No title resolvable ⇒ `null`. An id-shaped
   string is a plausible-looking title to whatever renders it, so the platform
   will not manufacture one; write your own fallback if you want one.
@@ -410,7 +417,7 @@ The extractor scans each body for known patterns and adds the matching capabilit
 | `*.object(…).insert / update / upsert / delete / patch / remove / create` | `api.write` |
 | `ctx.crypto.randomUUID` | `crypto.uuid` |
 | `ctx.log.info / warn / error / debug` | `log` |
-| `*.title(<argument>)` — the related-record form only; bare `ctx.title()` performs no read | `api.read` |
+| `*.title(<argument>)` — the related-record form only; bare `ctx.title()` names this record and needs no token | `api.read` |
 
 The matcher is deliberately over-inclusive — it names spellings the VM does not install (`patch`, `remove`, `get`, `list`, `create`) because an over-inferred token costs nothing the sandbox ever checks, while an under-inferred one surfaces as a sandbox refusal far from its cause. `create` is listed only for that reason: a body spelling it is refused at lowering (see [What the sandbox forbids](#what-the-sandbox-forbids)) and never reaches inference at all.
 

packages/cli/src/utils/extract-hook-body.ts

@@ -529,7 +529,8 @@ const CAPABILITY_PATTERNS: Array<{ rx: RegExp; cap: 'api.read' | 'api.write' | '
   { rx: /ctx\.log\.(?:info|warn|error|debug)\b/, cap: 'log' },
   // [#11293] `ctx.title(field)` — the RELATED-record form, and only that form.
   // `ctx.title()` resolves this record's title (formula included) from the
-  // state the hook is already firing on and performs no read at all, so it
+  // state the hook is already firing on — no read over its own columns, and
+  // only its declaration's one hop through a reference otherwise — so it
   // needs no capability and inferring one for it would tax the majority case
   // with a grant it never exercises. The argument form costs exactly one
   // `findOne` through the body's own read channel, which is the same read

packages/objectql/src/engine-formula-one-hop.test.ts

@@ -27,6 +27,7 @@
  */
 import { beforeEach, describe, expect, it } from 'vitest';
 import { ObjectQL } from './engine.js';
+import { resolveRecordTitle, resolveServedRecordTitle, servedRecordTitle } from './record-title.js';
 
 type Rec = Record<string, unknown>;
 
@@ -66,13 +67,13 @@ function makeDriver() {
     for (const f of fields) if (r[f] !== undefined) out[f] = r[f];
     return out;
   };
-  const calls: Array<{ object: string; ast: any }> = [];
+  const calls: Array<{ object: string; ast: any; options?: any }> = [];
   let n = 0;
   const driver: any = {
     name: 'memory', version: '0.0.0', supports: {},
     async connect() {}, async disconnect() {}, async checkHealth() { return true; }, async execute() { return null; },
-    async find(object: string, ast: any) {
-      calls.push({ object, ast });
+    async find(object: string, ast: any, options?: any) {
+      calls.push({ object, ast, options });
       const rows = Array.from(storeFor(object).values()).filter((r) => matches(r, ast?.where));
       // Hold the caller's page: `offset` then `limit`, as a real driver does.
       const offset = typeof ast?.offset === 'number' ? ast.offset : 0;
@@ -339,4 +340,84 @@ describe('#24197 — a formula field reads one hop through a reference', () => {
       engine.find('lib_loan', { orderBy: [{ field: 'title', order: 'asc' }], context: READER }),
     ).rejects.toMatchObject({ code: 'INVALID_SORT', status: 400, field: 'title' });
   });
+
+  // [#24197] The record a hook fires on was never read back, so nothing served
+  // its title: `serveFormulaField` is the read path's evaluation over a record
+  // in hand, reached through a channel (`ScopedContext`) that carries the
+  // reader's own context. The hook body's `ctx.title()` is built on it.
+  describe('serveFormulaField — a record in hand, served as a read serves it', () => {
+    const LOAN_SCHEMA = { name: 'lib_loan', nameField: 'title' };
+
+    it('reads the hop through the IN-HAND foreign keys, one batched read per reference field', async () => {
+      d.calls.length = 0;
+      // Stored l1 names b1; the in-flight state names b2.
+      const value = await engine.createContext(READER).serveFormulaField('lib_loan', { id: 'l1', book: 'b2', reader: 'r1' }, 'title');
+      expect(value).toBe('Emma · Ann');
+      expect(relatedReads('lib_book')).toBe(1);
+      expect(relatedReads('lib_reader')).toBe(1);
+      expect(relatedReads('lib_loan')).toBe(0);
+      expect(d.calls.find((c) => c.object === 'lib_book')!.ast.where).toEqual({ id: { $in: ['b2'] } });
+    });
+
+    it('answers the title a read of the same record answers, where the synchronous evaluator answers none', async () => {
+      const read = await engine.findOne('lib_loan', { where: { id: 'l3' }, context: READER });
+      const inHand = { id: 'l3', book: 'b3', reader: 'r2' };
+      expect(await resolveServedRecordTitle(LOAN_SCHEMA, inHand, engine.createContext(READER))).toBe(read?.title);
+      expect(read?.title).toBe('Ulysses · Bo');
+      // The before-state of this seam: no hop, no title.
+      expect(resolveRecordTitle({ ...LOAN_SCHEMA, fields: engine.getObject('lib_loan')!.fields }, inHand)).toBeUndefined();
+    });
+
+    it('a formula over the record\'s own columns reads nothing', async () => {
+      d.calls.length = 0;
+      expect(await engine.createContext(READER).serveFormulaField('lib_book', { id: 'b1', title: 'dune' }, 'shout')).toBe('DUNE');
+      expect(d.calls).toEqual([]);
+    });
+
+    it('a field that is not a formula answers undefined, so the stored column is read instead', async () => {
+      expect(await engine.createContext(READER).serveFormulaField('lib_loan', { id: 'l1', book: 'b1' }, 'book')).toBeUndefined();
+    });
+
+    it('reads the hop on the transaction the channel carries', async () => {
+      const trx = { __trx: 'body' };
+      d.calls.length = 0;
+      await engine.createContext({ ...READER, transaction: trx }).serveFormulaField('lib_loan', { id: 'l1', book: 'b1', reader: 'r1' }, 'title');
+      const hops = d.calls.filter((c) => c.object === 'lib_book' || c.object === 'lib_reader');
+      expect(hops).toHaveLength(2);
+      for (const c of hops) expect(c.options?.transaction).toBe(trx);
+    });
+
+    it('a related record the channel may not read leaves it null with the read\'s reason, and leaks nothing', async () => {
+      engine.registerMiddleware(async (ctx: any, next: () => Promise<void>) => {
+        if (ctx.object === 'lib_reader' && ['find', 'findOne'].includes(ctx.operation) && !ctx.context?.isSystem) {
+          const scoped = { visible: true };
+          ctx.ast = { ...(ctx.ast ?? {}), where: ctx.ast?.where ? { $and: [ctx.ast.where, scoped] } : scoped };
+        }
+        await next();
+      });
+      const inHand = { id: 'lx', book: 'b1', reader: 'r_hidden' };
+      expect(await engine.createContext(READER).serveFormulaField('lib_loan', inHand, 'title')).toBeNull();
+      expect(await resolveServedRecordTitle(LOAN_SCHEMA, inHand, engine.createContext(READER))).toBeUndefined();
+      expect(faultsOn('lib_loan', 'title')[0].message).toContain('was not found by this read');
+      // CONTROL — the SYSTEM channel reads the reader.
+      expect(await engine.createContext({ isSystem: true }).serveFormulaField('lib_loan', inHand, 'title')).toBe('Dune · Hidden Person');
+    });
+
+    it('a second hop reached through the related object\'s formula is not served', async () => {
+      registerLoan('record.book.shelf_code');
+      d.calls.length = 0;
+      expect(await engine.createContext(READER).serveFormulaField('lib_loan', { id: 'l1', book: 'b1' }, 'title')).toBeNull();
+      expect(relatedReads('lib_shelf')).toBe(0);
+    });
+
+    it('servedRecordTitle reads the title a read served off the row, never re-derived', async () => {
+      const book = { name: 'lib_book', nameField: 'shelf_code', fields: engine.getObject('lib_book')!.fields };
+      // A row a read returned with its one-hop title served…
+      expect(servedRecordTitle(book, { id: 'b1', title: 'Dune', shelf: 's1', shelf_code: 'A-1' })).toBe('A-1');
+      // …which the synchronous re-derivation throws away.
+      expect(resolveRecordTitle(book, { id: 'b1', title: 'Dune', shelf: 's1', shelf_code: 'A-1' })).toBeUndefined();
+      // A row the read served no title on has none.
+      expect(servedRecordTitle(book, { id: 'b1', title: 'Dune', shelf: 's1', shelf_code: null })).toBeUndefined();
+    });
+  });
 });

packages/objectql/src/engine.ts

@@ -2851,6 +2851,8 @@ async function hydrateWriteFormulas(
  * above. [#24197] For the same reason it preloads no relationship hop, so a
  * formula reading `record.<reference>.<field>` yields `null` here too. The read and write paths resolve the map first
  * ({@link ObjectQL.resolveFormulaPermissions}); a helper with no resolver cannot.
+ * A caller that holds a read channel and wants the read's answer calls
+ * {@link ObjectQL.serveFormulaField} (through `ScopedContext.serveFormulaField`).
  *
  * Returns `undefined` when `field` is not a declared formula field, which is
  * how a caller tells "read the stored column instead" from "the formula
@@ -6215,6 +6217,57 @@ export class ObjectQL implements IObjectQLEngine {
     return (row) => lookup(row);
   }
 
+  /**
+   * [#24197] Materialize ONE declared `formula` field against a record already
+   * in hand, SERVED the way a read serves it: the one hop the formula reads
+   * through a reference field ({@link resolveFormulaRelated}), the `can()`
+   * permission map ({@link resolveFormulaPermissions}) and the fault report
+   * ({@link formulaFaultSink}), all under `context`.
+   *
+   * The asynchronous twin of the module's {@link evaluateFormulaField}, which
+   * is synchronous and so serves none of the three. It exists for a record the
+   * engine never read back — a hook's in-flight state (stored ⊕ payload), whose
+   * foreign keys may name records the stored row does not — so the title a hook
+   * body asks for (`ctx.title()`) is the title a read of that state would show.
+   *
+   * ## Under which authority, on which connection
+   *
+   * `context` is the reader's own: every hop is the reader's own `find`, through
+   * every enforcement layer a read applies, exactly as the read path's hop is
+   * (`authority: 'caller'`). A related record that read does not return leaves
+   * the formula `null` with the read path's reason — never a fallback. A context
+   * carrying a `transaction` reads on that transaction's connection, so a body
+   * that opened one never asks the pool for a second (a deadlock on SQLite's
+   * `pool max=1`). Call it through {@link ScopedContext.serveFormulaField},
+   * which hands in the context the body's own `ctx.api` reads with.
+   *
+   * ## What it costs
+   *
+   * One batched read per reference field the formula reads through — the read
+   * path's own cost for one row — and nothing for a formula over the record's
+   * own columns. A read that is itself a formula's hop serves no hop of its own,
+   * so a second hop reads `null` with its reason here as it does on a read.
+   *
+   * Returns `undefined` when `field` is not a declared formula field on
+   * `objectName`, as {@link evaluateFormulaField} does. Evaluates against a
+   * shallow COPY of `record`.
+   */
+  async serveFormulaField(
+    objectName: string,
+    record: Record<string, unknown>,
+    field: string,
+    context?: ExecutionContext,
+  ): Promise<unknown> {
+    const schema = this._registry.getObject(objectName);
+    const { plan } = planFormulaProjection(schema, [field]);
+    if (plan.length === 0) return undefined;
+    const scratch: Record<string, unknown> = { ...record };
+    applyFormulaPlan(plan, [scratch], context, await this.resolveFormulaPermissions(
+      objectName, plan, [scratch], context, this.permissionResolution(context),
+    ), this.formulaFaultSink(objectName), await this.resolveFormulaRelated(schema, plan, [scratch], context));
+    return scratch[field];
+  }
+
   /**
    * [#22019] The (object, field) pairs whose formula fault this engine instance
    * has already reported — see {@link reportFormulaFault}.
@@ -21714,4 +21767,17 @@ export class ScopedContext implements IScopedContext, RunAsDerivableApi {
 
   /** Internal: expose the transaction handle for driver-level access */
   get transactionHandle() { return this.executionContext.transaction; }
+
+  /**
+   * [#24197] One declared `formula` field of `objectName`, evaluated over
+   * `record` as a read under THIS context serves it — {@link
+   * ObjectQL.serveFormulaField} with this context's own execution context, so
+   * the one hop the formula reads through a reference is read with this
+   * context's identity and permissions and, inside a transaction this context
+   * carries, on that transaction's connection. `undefined` when `field` is not
+   * a declared formula field.
+   */
+  async serveFormulaField(objectName: string, record: Record<string, unknown>, field: string): Promise<unknown> {
+    return (this.engine as unknown as ObjectQL).serveFormulaField(objectName, record, field, this.executionContext);
+  }
 }

packages/objectql/src/index.ts

@@ -402,11 +402,13 @@ export { hookRecordState } from './hook-wrappers.js';
 // exactly these; they are exported so it does not have to re-derive any of it.
 export {
   resolveRecordTitle,
+  resolveServedRecordTitle,
+  servedRecordTitle,
   resolveRelatedTitleTarget,
   titleFieldOf,
   RecordTitleFieldError,
 } from './record-title.js';
-export type { RelatedTitleTarget } from './record-title.js';
+export type { RelatedTitleTarget, FormulaServingChannel } from './record-title.js';
 export { evaluateFormulaField } from './engine.js';
 
 // Export Validation