Skip to content

Add typed JSON output to app security check - #8967

Open
jek wants to merge 3 commits into
app-security/support-json-schema-reviewfrom
app-security/support-json-schema-check
Open

jek wants to merge 3 commits into
app-security/support-json-schema-reviewfrom
app-security/support-json-schema-check

Conversation

@jek

@jek jek commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

WHY are these changes introduced?

app security check is the last app security command without --json. Its service also rendered output, prompted and set the exit code partway through, so a JSON mode couldn't be added without separating those.

WHAT is this pull request doing?

Adds --json to app security check. A scan returns:

{"status": "success", "valid": true, "app": {…}, "resultsDirectory": "…",
 "deterministicFindingsPath": "…/deterministic-findings.json", "agentChecksPath": "…/agent-checks.json",
 "scope": {"includeDirs": [], "excludes": [], "noGitIgnore": false}, "scanDirectories": [{"directory": "…", "origin": "app-directory"}],
 "agentCheckCount": 36, "blocking": {"level": "none", "blockedCheckCount": 0},
 "instructions": {"content": "<Markdown>", "copied": true}}
  • No findings. The result doesn't include them; review --json is the one place findings are reported. valid and blockedCheckCount follow --blocking, with the same rule and exit codes as today.
  • Instructions. In a terminal, the instructions prompt is still offered under --json, on stderr. Its choices say that the instructions go into the result. --print-instructions includes them without asking. With --skip-instructions, "Nothing", or --no-input, instructions is left out.
  • --list-files --json returns {status, app, scope, scanDirectories, files: [{path}]} with absolute paths.
  • Declining "Scan it without app configuration?" prints {"status": "cancelled"} and exits 0. Text mode still exits 1 with the same error.
  • Stderr events. Gathering warnings and the "To skip these prompts next time" tip become events on stderr.

Commits:

  1. securityCheck returns a listed, scanned or cancelled outcome, and a presenter in security-check/result.ts renders it, prompts for instructions and sets the exit code. Declining is an outcome of the selection rather than an error. No behaviour change.
  2. When the instructions can't be copied to the clipboard, check warns instead of failing: the scan has finished, so the exit code still follows --blocking. It has its own patch changeset, which also narrows the clipboard changeset from earlier in the stack.
  3. Add --json, and remove the App Security section from the JSON output lint exceptions.

How to manually test your changes?

Build once and run the local CLI directly, so nothing else reaches stdout:

pnpm nx build cli
alias shopify-dev="node $PWD/packages/cli/bin/dev.js"

Run these in a real terminal with stdout redirected, so the prompts show while the result goes to the file:

# An app with two app configuration files: the picker and the instructions prompt appear; out.json holds one object
shopify-dev app security check --path <app> --json > out.json; echo "exit $?"; cat out.json

# A directory without app configuration: answer "No" -> {"status": "cancelled"}, exit 0
shopify-dev app security check --path <empty-dir> --json > out.json; echo "exit $?"; cat out.json

shopify-dev app security check --path <app> --config <name> --json --list-files
shopify-dev app security check --path <app> --config <name> --json --no-input --blocking low; echo "exit $?"   # valid: false, exit 1 when there are findings

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is user-facing — I've identified the correct bump type (patch for bug fixes · minor for new features · major for breaking changes) and added a changeset with pnpm changeset add

@jek
jek added this pull request to stack #8969 October 21, 2026 00:00
@github-actions github-actions Bot added the Area: @shopify/cli @shopify/cli package issues label Oct 21, 2026
@jek jek changed the title app security/support json schema check Add typed JSON output to app security check Oct 21, 2026
jek added 2 commits October 20, 2026 17:40
`securityCheck` now returns what it did instead of rendering it: the
listed files, the finished scan with its `--blocking` outcome, or a
cancellation when the user declines to scan without app configuration.
It still records the telemetry. The command passes the outcome to a
presenter in security-check/, which renders the gathering warnings and
the report, offers and delivers the coding-agent instructions, and sets
exit code 1 when `--blocking` is breached. The tip that repeats a
prompted selection is still shown before the scan, through a callback
that the command passes in.

`resolveAppSecuritySelection` returns declining to scan without app
configuration as an outcome instead of aborting, so that `--json` can
report it. In the terminal the presenter aborts with the same error as
before. The text output, prompts and exit codes don't change.
Copying the coding-agent instructions is the last step of `check`,
after the scan finished and its report is shown. A clipboard that isn't
available aborted the command, so the exit code no longer reported the
scan. `check` now warns with the clipboard tool's message and suggests
`--print-instructions` instead, and the exit code follows `--blocking`.

`instructions --copy` still fails, because copying is all it does.
@jek
jek force-pushed the app-security/support-json-schema-check branch from 5bc0499 to d42f50b Compare October 21, 2026 00:46
`--json` prints one result for the scan: the app, the results directory
and the two files that the scan replaced, its scope and scan
directories, the number of agent checks, and whether `--blocking`
passed. A breach still exits 1, after the result is printed. The result
holds no findings: `review --json` shows them. Paths are absolute, with
native separators.

In a terminal the instructions prompt is still offered, on stderr, with
choices to include the instructions in the result, and to copy them as
well by default. `--print-instructions` includes them without asking. A
failed copy warns on stderr and reports `copied: false`. `--list-files`
lists the gathered files with absolute paths. Declining to scan without
app configuration prints a cancelled result and exits 0. Gathering
warnings and the tip to skip the prompts are events on stderr.

`check` was the last app security command without typed JSON output, so
the exceptions list drops its app security section.
@jek
jek force-pushed the app-security/support-json-schema-check branch from d42f50b to b9edbe4 Compare October 21, 2026 01:11
@jek
jek marked this pull request as ready for review October 21, 2026 13:58
@jek
jek requested review from a team as code owners October 21, 2026 13:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: @shopify/cli @shopify/cli package issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant