Skip to content

test(publish-smoke): build the scaffolded project, and assert its identity first - #8968

Merged
os-project-manager merged 1 commit into
mainfrom
claude/issue-8678-registry-canary-runs-build
Aug 16, 2026
Merged

os-project-manager merged 1 commit into
mainfrom
claude/issue-8678-registry-canary-runs-build

Conversation

@os-project-manager

Copy link
Copy Markdown
Collaborator

Fixes #8678

scripts/publish-smoke.sh went straight from npm install to objectstack dev. npm run build — the second command the scaffolder prints — was never run, so the gate was structurally unable to see the one failure class it exists to catch.

The evidence that this was a real blind spot

Every "published scaffold is broken" incident so far surfaced at build, and at build only:

Incident What broke scaffold install build
#4902 all five remote templates at once 0 0 2
#7644 the namespace rewrite on 16.1.0 0 0 2
#8677 retired enable.trash / enable.mru on 17.0.0 0 0 2

Scheduled run #1932 concluded SUCCESS on 2026-08-10 against a published create-objectstack whose scaffold failed its first build on four templates. That green was honest, and it meant nothing — which is the definition of a vacuous gate.

What the new section 1b does

1. Asserts the project's identity before building. This is the part that makes the gate capable of failing, and it is not cosmetic. The scaffolder's namespace rewrite only does anything when the project namespace differs from the template's own — and #7644 was precisely a rewrite that silently did nothing. Measured both ways against the #7644 state (create-objectstack@16.1.0):

  • project namespace ≠ template namespace → build reports 4 missing the package namespace prefix errors, exits 2
  • project namespace = template namespace → the same defect yields 0 such errors, and the whole class goes invisible

APP_NAME differing from the template namespace was accidental before this PR. Now it is asserted, with a failure message that names the remedy.

2. Runs the build, with the package manager that owns that project's node_modules (pnpm in pack mode, npm in registry mode).

3. Refuses a build that emits nothing. Exit 0 alone is not the assertion — a build producing no dist/objectstack.json would satisfy it while leaving objectstack start with no artifact to serve. That is a false green of exactly the kind this gate is supposed to stop, so it is checked separately.

4. Makes absence loud. If the template stops shipping a build script, the run would otherwise fail with a package-manager "missing script" message that says nothing about the template. It now names the real remedy instead.

5. Splits the failure issue by owner. The canary's auto-filed issue previously attributed every failure to the #3091 ^-range class. It now tells the reader to check which step failed, because a build failure is a create-objectstack bug and an auth/CRUD failure is the dependency-drift bug — different owners, different fixes.

Cost

Measured on GA 17.0.0 (Node 22.22.2 / npm 10.9.7): scaffold 2s, install 60s, build 3s. The install was already being paid for, so the added cost of this coverage is ~3s against a job budgeted 25 minutes.

One assumption in the card that no longer holds

#8678 weighed this against the cost of "five extra installs". That concern is moot: the five remote templates were delisted and are unmaintained, and the scaffolder's catalog is blank alone (packages/create-objectstack/src/template-registry.ts). Scope here is blank for the same reason — it is the entire supported surface, and the whole first-run experience for a new user.

Scope

Two files, both CI infrastructure: .github/workflows/publish-smoke.yml (header comment + the auto-filed issue body) and scripts/publish-smoke.sh (section 1b). No published package source changes, so no changeset — skip-changeset.


Generated by Claude Code

…ntity

The publish smoke went straight from install to `objectstack dev`, so it never
ran `npm run build` in either mode. That is where every "published scaffold is
broken" incident so far actually surfaced (#4902, #7644, #8677): in all of them
scaffold and install exited 0 and only the build exited 2, so the gate was
structurally unable to see them. Scheduled registry-canary run #1932 concluded
success on 2026-08-10 against a create-objectstack whose scaffold failed first
build on four templates.

Both modes now run the project's own build script and assert exit 0, plus a
non-empty dist/objectstack.json so a build that emits nothing cannot read as a
pass. Scope is the bundled `blank` template — the scaffolder's whole catalog
since the five remote templates were delisted.

Also assert the scaffolded namespace differs from the template's own. The
identity rewrite only does anything when the two differ, and #7644 was a
rewrite that silently did nothing: measured against create-objectstack@16.1.0,
the build reports 4 namespace-prefix errors when they differ and 0 when the
project is named after the template. `smoke-app` differing from `blank` was
accidental; it is asserted now.

Measured cost on GA 17.0.0: build 3s on top of an install already being paid.
@vercel

vercel Bot commented Aug 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 16, 2026 3:15am

Request Review

@os-project-manager os-project-manager added tests tooling skip-changeset PR has no user-facing published change; bypasses the changeset gate labels Aug 16, 2026 — with Claude
@os-project-manager
os-project-manager marked this pull request as ready for review August 16, 2026 03:18
@os-project-manager
os-project-manager added this pull request to the merge queue Aug 16, 2026
Merged via the queue into main with commit cdc0495 Aug 16, 2026
32 checks passed
@os-project-manager
os-project-manager deleted the claude/issue-8678-registry-canary-runs-build branch August 16, 2026 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants