fix(objectql): give a runtime-authored object a metadata-service copy on a kernel with an environmentId, so both MCP doors refuse an object whose API exposure is off

.changeset/23689-env-kernel-runtime-object-copy.md

@@ -0,0 +1,21 @@
+---
+'@objectstack/objectql': patch
+---
+
+fix(objectql): on a kernel with an `environmentId` (`os dev`, `os serve`, `os start`, the cloud single-environment runtime), a runtime-authored object now has a metadata-service copy, so both MCP doors refuse an object whose API exposure is off
+
+Clause-②: no
+
+On a kernel built with an `environmentId`, the `metadata` service is sourced from the artifact and the boot bridge does not run. Until now, the copy follow that keeps a runtime-authored object's copy current also returned early on that shape. So the service held **no** copy of **any** object saved through Studio, the REST metadata door or an AI author, and the MCP exposure gates read that absence as "no declaration". Measured on the real `os dev` (`env_local`), where the REST data door refused the same objects with `OBJECT_API_DISABLED`:
+
+- **Saved with `apiEnabled: false`, or turned off after a save.** The HTTP MCP door served `query_records` and `create_record`, and the create wrote a row. The MCP stdio door did the same.
+- **Saved with `apiEnabled: false`, then a restart.** Still served: the hydrated object had no copy, and no save fired after boot.
+- **Exposed objects.** `list_objects` did not name them, and `describe_object` answered "not found".
+
+The fix is at the copy's producer, and it bounds what is written the same way the no-`environmentId` shape already does:
+
+- **The copy follow runs on every kernel shape.** A save refreshes the copy before it returns, through `registerInMemory` (in memory, no persistence, no announce). A delete removes it once the stored row is gone and the registry no longer serves the name.
+- **A kernel that hydrates its own `sys_metadata` (`hydrateMetadataFromDb`) copies its hydrated runtime-authored objects once at boot.** That option is set only on a kernel whose registry is its own. Only objects whose registry owner row is tenant-authored are copied, so a code package's objects still reach the service from the artifact alone. The full boot bridge stays off on this shape.
+- **A boot copy that cannot land is logged at `error`.** The line names the objects, the consequence (the MCP exposure gates serve them even when their exposure is off) and the way out.
+
+Both exposure gates read this service: the MCP stdio door's `enforceApiExposure`, and `refuseUnexposedDataAction`, which the HTTP MCP data verbs run through. Both now answer `OBJECT_API_DISABLED` / 404 for such an object, which is what `apiEnabled: false` declares. Neither gate's fall-open rule for an absent definition changes. The kernel shape without an `environmentId` is unchanged.

packages/objectql/src/plugin-runtime-object-metadata-copy.test.ts

@@ -46,7 +46,7 @@
  */
 
 import { randomUUID } from 'node:crypto';
-import { mkdtempSync, rmSync } from 'node:fs';
+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { afterEach, describe, expect, it, vi } from 'vitest';
@@ -320,3 +320,132 @@ describe('the metadata service\'s copy of a runtime-authored object follows the
         expect((await copyOf(k, OTHER)).copy).toMatchObject({ enable: { apiEnabled: true } });
     });
 });
+
+/**
+ * The same copy on a kernel with an `environmentId` that hydrates its own
+ * `sys_metadata`: `createStandaloneStack`'s composition (`os dev` as
+ * `env_local`, `os serve`, `os start`) and the cloud single-environment
+ * runtime. Its `metadata` service is sourced from the artifact and the boot
+ * bridge does not run, so before the fix a runtime-authored object had NO copy
+ * there, exposed or not, and both MCP exposure gates served one saved with
+ * `apiEnabled: false` (measured through the real `os dev`).
+ */
+describe('on a kernel with an environmentId, a runtime-authored object has a copy that follows the registry', () => {
+    const ENV = 'env_local';
+    const live: ObjectKernel[] = [];
+    const roots: string[] = [];
+    afterEach(async () => {
+        while (live.length) {
+            const kernel = live.pop()!;
+            if (kernel.getState() === 'running') await kernel.shutdown();
+        }
+        while (roots.length) rmSync(roots.pop()!, { recursive: true, force: true });
+    });
+
+    /**
+     * One boot over `stores`, with `createStandaloneStack`'s two options:
+     * `MetadataPlugin` reading an (empty) artifact under the environment, and
+     * `ObjectQLPlugin` with the `environmentId` and `hydrateMetadataFromDb`.
+     */
+    async function boot(stores: Stores, beforeBoot?: (engine: ObjectQL) => void): Promise<Booted> {
+        const kernel = new ObjectKernel({ logger: { level: 'silent' }, gracefulShutdown: false });
+        live.push(kernel);
+        const root = mkdtempSync(join(tmpdir(), 'os-23689-'));
+        roots.push(root);
+        const artifact = join(root, 'objectstack.json');
+        writeFileSync(artifact, JSON.stringify({
+            manifest: { id: 'com.qa.empty', name: 'empty', version: '0.0.0', type: 'app' },
+            objects: [], views: [], apps: [], flows: [], requires: [],
+        }));
+        const engine = new ObjectQL();
+        engine.registerDriver(makeMemoryDriver(stores), true);
+        await kernel.use(new MetadataPlugin({
+            watch: false, artifactWatch: false, environmentId: ENV,
+            artifactSource: { mode: 'local-file', path: artifact }, rootDir: root,
+        }));
+        await kernel.use(new ObjectQLPlugin({ ql: engine, environmentId: ENV, hydrateMetadataFromDb: true }));
+        beforeBoot?.(engine);
+        await kernel.bootstrap();
+        return { kernel, engine, protocol: kernel.getService('protocol'), metadata: kernel.getService('metadata') };
+    }
+
+    const save = (k: Booted, name: string, label: string, enable?: Record<string, unknown>) =>
+        k.protocol.saveMetaItem({ type: 'object', name, item: objectBody(name, label, enable) });
+
+    async function copyOf(k: Booted, name: string) {
+        const listed = ((await k.metadata.listObjects()) as Array<{ name?: string }>).find((o) => o?.name === name);
+        return { copy: await k.metadata.getObject(name), listed };
+    }
+
+    const withShippedPackage = (engine: ObjectQL) => {
+        engine.registerApp({
+            id: SHIPPED_PKG, name: SHIPPED_PKG, version: '1.0.0', type: 'app',
+            objects: [objectBody(SHIPPED, 'Shipped')],
+        });
+    };
+
+    it('created after boot with exposure off: a copy at once, carrying its declaration', async () => {
+        const k = await boot(new Map());
+        expect((await save(k, NAME, 'Gadget', { apiEnabled: false })).success).toBe(true);
+
+        const { copy, listed } = await copyOf(k, NAME);
+        expect(copy).toMatchObject({ label: 'Gadget', enable: { apiEnabled: false }, _provenance: 'org' });
+        expect(listed).toBeDefined();
+    });
+
+    it('saved exposed, then turned off: the copy says so before the save returns', async () => {
+        const k = await boot(new Map());
+        expect((await save(k, NAME, 'Gadget', { apiEnabled: true })).success).toBe(true);
+        expect((await save(k, NAME, 'Gadget Q', { apiEnabled: false })).success).toBe(true);
+
+        expect((await copyOf(k, NAME)).copy)
+            .toMatchObject({ label: 'Gadget Q', enable: { apiEnabled: false }, _provenance: 'org' });
+    });
+
+    it('saved with exposure off, then a restart: the boot gives the hydrated object its copy (no mutation fires for it)', async () => {
+        const stores: Stores = new Map();
+        const k1 = await boot(stores);
+        expect((await save(k1, NAME, 'Gadget', { apiEnabled: false })).success).toBe(true);
+        expect((await save(k1, OTHER, 'Other', { apiEnabled: true })).success).toBe(true);
+        await k1.kernel.shutdown();
+
+        const k2 = await boot(stores);
+        expect(k2.engine.registry.getObject(NAME)).toMatchObject({ enable: { apiEnabled: false } });
+        expect((await copyOf(k2, NAME)).copy)
+            .toMatchObject({ label: 'Gadget', enable: { apiEnabled: false }, _provenance: 'org' });
+        expect((await copyOf(k2, OTHER)).copy).toMatchObject({ enable: { apiEnabled: true } });
+    });
+
+    it('deleted: the copy is gone and the listing no longer names it', async () => {
+        const stores: Stores = new Map();
+        const k1 = await boot(stores);
+        expect((await save(k1, NAME, 'Gadget', { apiEnabled: true })).success).toBe(true);
+        await k1.kernel.shutdown();
+        const k = await boot(stores);
+        expect((await copyOf(k, NAME)).copy).toBeDefined();
+
+        expect(await k.protocol.deleteMetaItem({ type: 'object', name: NAME })).toMatchObject({ success: true, reset: true });
+
+        await vi.waitFor(async () => {
+            const { copy, listed } = await copyOf(k, NAME);
+            expect(copy).toBeUndefined();
+            expect(listed).toBeUndefined();
+        });
+    });
+
+    it('a code package\'s object gets no copy from the boot pass or from a runtime edit beside it', async () => {
+        const stores: Stores = new Map();
+        const k1 = await boot(stores, withShippedPackage);
+        expect((await save(k1, NAME, 'Gadget', { apiEnabled: true })).success).toBe(true);
+        await k1.kernel.shutdown();
+
+        const k = await boot(stores, withShippedPackage);
+        expect(k.engine.registry.getObject(SHIPPED)).toMatchObject({ label: 'Shipped' });
+        // The code package's objects reach this service from the artifact, never from this pass.
+        expect(await k.metadata.getObject(SHIPPED)).toBeUndefined();
+        expect((await save(k, NAME, 'Gadget Q', { apiEnabled: false })).success).toBe(true);
+        await new Promise((resolve) => setTimeout(resolve, 150));
+        expect(await k.metadata.getObject(SHIPPED)).toBeUndefined();
+        expect((await copyOf(k, NAME)).copy).toMatchObject({ enable: { apiEnabled: false } });
+    });
+});

packages/objectql/src/plugin.ts

@@ -1068,6 +1068,10 @@ export class ObjectQLPlugin implements Plugin {
     // skip it in that case.
     if (this.environmentId === undefined) {
         await this.bridgeObjectsToMetadataService(ctx);
+    } else if (this.hydrateMetadataFromDb) {
+        // The runtime-authored objects the hydration above just put in this
+        // kernel's own registry. See {@link bridgeRuntimeObjectsToMetadataService}.
+        await this.bridgeRuntimeObjectsToMetadataService(ctx);
     }
     // Arm the incremental per-manifest bridge for everything after start() —
     // marketplace install / rehydrate register through the `manifest`
@@ -2397,6 +2401,56 @@ export class ObjectQLPlugin implements Plugin {
     }
   }
 
+  /**
+   * The boot half of {@link followRuntimeObjectCopy} on a kernel with an
+   * `environmentId` that hydrates its own `sys_metadata`
+   * (`hydrateMetadataFromDb`: `os dev`, `os serve`, `os start` and the cloud
+   * single-environment runtime): copy each RUNTIME-AUTHORED object the
+   * hydration put in the registry into the `metadata` service.
+   *
+   * The full {@link bridgeObjectsToMetadataService} stays off on this shape
+   * (cross-project leakage, and the code packages' objects come from the
+   * artifact). But nothing else gives a hydrated runtime object a copy, and no
+   * mutation fires for it after boot: an object saved with `apiEnabled: false`
+   * and then restarted was served by both exposure gates, which read an absent
+   * copy as "no declaration". `hydrateMetadataFromDb` is set only on a kernel
+   * whose registry is its own, so every object this pass copies is one this
+   * environment authored. Same write as the follow: tenant-authored owner rows
+   * only, through `registerInMemory` (no persistence, no announce).
+   */
+  private async bridgeRuntimeObjectsToMetadataService(ctx: PluginContext): Promise<void> {
+    if (!this.ql?.registry) return;
+    let metadataService: IMetadataService | undefined;
+    try {
+      metadataService = ctx.getService<IMetadataService>('metadata');
+    } catch {
+      return; // no metadata service on this kernel — there is no copy to keep
+    }
+    if (!metadataService) return;
+    const failed: string[] = [];
+    let copied = 0;
+    for (const obj of this.ql.registry.getAllObjects()) {
+      try {
+        if (await this.mirrorRuntimeObject(metadataService, obj.name)) copied++;
+      } catch {
+        failed.push(obj.name);
+      }
+    }
+    if (copied > 0) {
+      ctx.logger.debug('Copied runtime-authored objects into the metadata service', { count: copied });
+    }
+    if (failed.length > 0) {
+      ctx.logger.error(
+        `[ObjectQLPlugin] the metadata service holds no copy of runtime-authored object(s) ${failed.map((n) => `'${n}'`).join(', ')}: `
+        + 'the registry, the data door and REST serve them, but the MCP exposure gates read an absent copy as "no '
+        + 'declaration" and serve them even when their API exposure is off, and MCP does not list them. Fix: restart '
+        + 'this process; if it recurs, check that the metadata service accepts in-memory registration.',
+        undefined,
+        { objects: failed },
+      );
+    }
+  }
+
   /**
    * Keep the metadata service's copy of a RUNTIME-AUTHORED object current after
    * boot, and take it out when the object is deleted.
@@ -2439,12 +2493,17 @@ export class ObjectQLPlugin implements Plugin {
    *    Afterwards the registry is asked again: a re-creation that landed while
    *    the removal was awaiting its loaders has its copy put back.
    *
-   * Armed on the kernel shape the bridge runs on (no `environmentId`): a
-   * project-scoped kernel's service is sourced from the artifact and holds no
-   * copy of the registry to keep current.
+   * Armed on every kernel shape. On a kernel with an `environmentId` (`os dev`
+   * as `env_local`, the cloud single-environment runtime) the service is
+   * sourced from the artifact and the boot bridge does not run, so a
+   * runtime-authored object had no copy there at all: both exposure gates read
+   * that absence as "no declaration" and served an object saved with
+   * `apiEnabled: false`. The copy written here is the object THIS kernel's
+   * authoring door just saved, so the cross-project leakage the full bridge is
+   * gated against does not arise. Its boot half on that shape is
+   * {@link bridgeRuntimeObjectsToMetadataService}.
    */
   private followRuntimeObjectCopy(ctx: PluginContext, evt: { name: string; state: string }): void {
-    if (this.environmentId !== undefined) return;
     if (typeof evt?.name !== 'string' || evt.name === '') return;
     // An async function runs synchronously up to its first `await`: the
     // refresh of a served runtime object is done before this call returns.

packages/qa/dogfood/test/mcp-stdio-runtime-object-exposure.test.ts

@@ -49,7 +49,7 @@
  */
 
 import { randomUUID } from 'node:crypto';
-import { mkdtempSync, rmSync } from 'node:fs';
+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
@@ -177,7 +177,22 @@ function inProcessTransport() {
   return { transport, rpc, notify: (method: string) => transport.onmessage?.({ jsonrpc: '2.0', method }) };
 }
 
-describe('the MCP stdio door follows a runtime-authored object\'s current definition', () => {
+/**
+ * The two kernel shapes the door is pinned on. `none`: an `ObjectQLPlugin` with
+ * no `environmentId`, where the boot bridge copies the registry. `env_local`:
+ * `createStandaloneStack`'s composition (`os dev`, `os serve`, `os start`, and
+ * the cloud single-environment runtime's shape) — an artifact-sourced
+ * `metadata` service and an `ObjectQLPlugin` with the `environmentId` and
+ * `hydrateMetadataFromDb`, where no bridge runs. On that shape the service held
+ * no copy of ANY runtime-authored object before the fix, so the door served
+ * every one of them whatever its exposure (measured through the real `os dev`).
+ */
+const SHAPES = [
+  { shape: 'no environmentId', environmentId: undefined },
+  { shape: 'environmentId env_local (createStandaloneStack)', environmentId: 'env_local' },
+] as const;
+
+describe.each(SHAPES)('the MCP stdio door follows a runtime-authored object\'s current definition — $shape', ({ environmentId }) => {
   const live: ObjectKernel[] = [];
   const roots: string[] = [];
   const originalKey = process.env.OS_MCP_STDIO_API_KEY;
@@ -205,9 +220,9 @@ describe('the MCP stdio door follows a runtime-authored object\'s current defini
 
   /**
    * One boot over `stores`, then a stdio session over it: `MetadataPlugin` provides
-   * the `metadata` service, `ObjectQLPlugin` with no `environmentId` hydrates
-   * `sys_metadata` and runs the boot bridge, and `MCPServerPlugin` builds the
-   * stdio data bridge over this kernel's own services.
+   * the `metadata` service, `ObjectQLPlugin` (in the shape under test) hydrates
+   * `sys_metadata`, and `MCPServerPlugin` builds the stdio data bridge over this
+   * kernel's own services.
    */
   async function boot(stores: Stores): Promise<Booted> {
     const kernel = new ObjectKernel({ logger: { level: 'silent' }, gracefulShutdown: false });
@@ -216,8 +231,21 @@ describe('the MCP stdio door follows a runtime-authored object\'s current defini
     roots.push(root);
     const engine = new ObjectQL();
     engine.registerDriver(makeMemoryDriver(stores), true);
-    await kernel.use(new MetadataPlugin({ watch: false, rootDir: root }));
-    await kernel.use(new ObjectQLPlugin({ ql: engine }));
+    if (environmentId === undefined) {
+      await kernel.use(new MetadataPlugin({ watch: false, rootDir: root }));
+      await kernel.use(new ObjectQLPlugin({ ql: engine }));
+    } else {
+      const artifact = join(root, 'objectstack.json');
+      writeFileSync(artifact, JSON.stringify({
+        manifest: { id: 'com.qa.empty', name: 'empty', version: '0.0.0', type: 'app' },
+        objects: [], views: [], apps: [], flows: [], requires: [],
+      }));
+      await kernel.use(new MetadataPlugin({
+        watch: false, artifactWatch: false, environmentId,
+        artifactSource: { mode: 'local-file', path: artifact }, rootDir: root,
+      }));
+      await kernel.use(new ObjectQLPlugin({ ql: engine, environmentId, hydrateMetadataFromDb: true }));
+    }
     await kernel.bootstrap();
 
     // The faked seam, and only it: the key row the stdio principal resolves from.
@@ -360,6 +388,20 @@ describe('the MCP stdio door follows a runtime-authored object\'s current defini
     expect(stores.get(GATED)?.size).toBe(1);
   });
 
+  it('saved with exposure off, then a restart: refused from the first call after the boot', async () => {
+    const stores: Stores = new Map();
+    const k1 = await boot(stores);
+    expect((await save(k1, GATED, 'Gated', { apiEnabled: false })).success).toBe(true);
+    await k1.engine.insert(GATED, { name: 'rec-1' }, SYSTEM);
+    await k1.kernel.shutdown();
+
+    const k = await boot(stores);
+    expect(refusal(await k.call('query_records', { objectName: GATED }))).toEqual({ code: 'OBJECT_API_DISABLED', status: 404 });
+    expect(refusal(await k.call('create_record', { objectName: GATED, data: { name: 'leak-probe' } })))
+      .toEqual({ code: 'OBJECT_API_DISABLED', status: 404 });
+    expect(stores.get(GATED)?.size).toBe(1);
+  });
+
   it('deleted: it is neither listed nor described, and its data verbs are refused', async () => {
     const stores: Stores = new Map();
     const k = await bootHydrated(stores);