Skip to content

feat(host): back up hosted and compute channels and restore backups as new channels - #435

Merged
iamnbutler merged 3 commits into
mainfrom
hosted-backup
Oct 15, 2026
Merged

iamnbutler merged 3 commits into
mainfrom
hosted-backup

Conversation

@iamnbutler

Copy link
Copy Markdown
Contributor

Fixes #178.

ace backup <channel> <dir> now works for hosted and cloud compute channels as well as local ones (option B: export-based backups). ace restore <dir> adds any backup as a new local channel.

Design

  • Consistency without a freeze. The service adds a backup action to the existing POST /channels/:id/transfer. It runs backupSnapshot, which reads every pi table in one db.transaction. All pi reads and writes on a hosted channel go through the same DurableSqlite, which runs that transaction on its own inside storage.transaction. So the snapshot is a consistent point in time without freeze: clients stay connected and runs keep going. Nothing is frozen, so a failed backup can't leave the channel frozen. active and frozen channels can be backed up. An inactive imported copy from an unfinished move is refused.
  • Unfinished work is kept. Unlike exportSnapshot for moves, a backup doesn't refuse a busy channel. It records unfinished runs, as a local VACUUM INTO snapshot does.
  • Format and limits. snapshot.json is the same version 1 transfer snapshot, with its SHA-256 digest of the canonical encoding. backup.json (format 1) adds the service URL and that digest next to the per-file SHA-256 checksums. Over 32 MiB is refused with "The channel is larger than the 32 MiB snapshot limit, so it cannot be moved or backed up".
  • Auth and refusal. The host uses hostedAuth() (the team's ACE_SECRET) and the service's existing bearer check. A service without the action answers 400 "Unknown transfer action". The host reports that as "<url> predates hosted channel backups; redeploy it to back this channel up." The service is asked before the destination is created, so a refusal leaves nothing behind.
  • Restore. ace restore <backup> [--project <dir>] checks every file against backup.json, then creates a new local channel with a new ID and a new lane prefix. A hosted snapshot goes through importSnapshot, which verifies the digest and refuses a non-empty store. A local backup's channel.sqlite is copied. The catalog record is written last. The copy can never run under the original's ID, whether that channel is still live locally or on its service. The in-place same-ID recovery for local channels in docs/data.md is unchanged. A hosted channel is never replaced in place.
  • Scope. Workspace files are out of scope. A compute channel's files are covered by the checkpoint (Preserve and recover cloud workspace files beyond the pilot checkpoint limit #413), and docs/data.md says so.
  • transfer.ts now treats local wrangler dev's __miniflare_do_name table as internal. Without this, local exports (including moves) were refused with "unknown tables". It doesn't affect deployed Workers.

Redeploy needed: the backup transfer action is new. Both services/channel (ace-channel) and services/compute (ace-compute-pilot, which serves the same HostedChannel) need redeploying before ace backup works against them. Until then, hosts report the redeploy message above. I did not deploy anything.

Verified locally

  • bun types and bun run ci pass.
  • Setup: local wrangler dev of services/channel and services/compute (containers off). The source host ran ace serve --loopback with its own ACE_HOME, keychain service and a stub tailscale. Runs used real openai/gpt-5-mini. No request went to a deployed Worker.
  • Channels: I created a hosted channel (this host as workspace) and a compute channel (through compute.create, i.e. /workspaces/:id/init). Each got a message and a real model run, and the hosted one also got a bash tool run.
  • Backups: ace backup succeeded on both while they were active. A third backup, taken mid-run on the hosted channel, took about 1.6 s. It recorded tasks waiting/running while the live run finished, and a message posted during that run was admitted.
  • Restore: ace restore on each made new local channels. Their pi tables re-hashed to the backup digests (counts matched, e.g. 8 entries, 4 tasks, 10 revisions).
    • ace watch replayed the same history, including the tool call and result.
    • A real model in each restored copy answered from that history ("heliotrope", the bash output "hi").
    • The mid-run copy reported the interrupted bash tool and reran it to completion.
    • A local channel's backup also restored as a new channel with its message.
  • Live channels afterwards: both still answered real model runs. A thaw request returned "Only a frozen channel can be thawed (state: active)" for both, so neither was frozen.
  • Older service: with the previous object.ts, ace backup printed the redeploy message and created no directory.

Not verified

  • The 32 MiB refusal wasn't exercised with a channel that large.
  • No container ran, so the compute channel's workspace wasn't connected. Its pi state is the same HostedChannel.

Built in Ace

No — built with Claude Code outside Ace by an overnight issue watcher, because this Mac's Ace Canary is a participant in the running two-person study (#218); dogfooding gap tracked in #5.

…s new channels

`ace backup` now takes a hosted or cloud compute channel's pi state from its service with a new
`backup` transfer action, which reads one consistent snapshot without freezing the channel.
`ace restore` verifies a backup and adds it as a new local channel with a new ID and lane prefix.

Why: packages/channel/src/transfer.ts#function backupSnapshot | the read stays one transaction on the channel's DurableSqlite, which serializes every pi read and write, so a backup needs no freeze
Why: apps/host/src/backup.ts#function restore | a restored copy always gets a new ID and lane prefix so it can never run as the original channel
Why: apps/host/src/backup.ts#function exported | the service is asked before the destination exists, so a refusal or an older service leaves nothing behind
@iamnbutler
iamnbutler merged commit bbdc04d into main Oct 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Back up and restore hosted channels

1 participant