Skip to content

feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request - #99

Merged
iamnbutler merged 1 commit into
mainfrom
feat/insert-attachments
Oct 17, 2026
Merged

iamnbutler merged 1 commit into
mainfrom
feat/insert-attachments

Conversation

@iamnbutler

Copy link
Copy Markdown
Contributor

Part of #3. desktop_insert can now paste an image or files through a temporary clipboard, with the same restore rules as text.

Request

  • format: "image" with one absolute path (PNG, JPEG or TIFF, at most 10 MiB and 64 MP; original bytes and type are kept), or format: "files" with 1–32 absolute paths.
  • No text or route in that form.
  • Optional clipboard: "restore" (the default) or "replace".

Route, fixed before any input

  • Guarded mode (restore) pastes only into a focused AXTextArea that meets both conditions:
    • its ancestry proves native text (TextInputRoute.resolve(container:) == .nativeAX, so not web content or Electron);
    • its complete text and selection can be read.
  • Any other receiver is refused before the clipboard is touched, with PASTE_UNVERIFIABLE_RECEIVER. The hint offers clipboard: "replace", or desktop_clipboard_write followed by Command-V.

Observed consumption

  • Expected edit: the selection is replaced by one U+FFFC per item, and the caret sits after them.
  • If AXAttributedStringForRange marks any of those characters with AXAttachment, it must mark all of them (attachment_evidence: "attachment_attribute"). If it marks none, or can't be read, the replacement characters and caret decide ("replacement_characters"). A partly marked range does not count.
  • After that, the text paste rules apply unchanged:
    • the reservation is taken before V goes down;
    • the edit is polled for 2 s, and when it appears the reservation is released and the clipboard restored (generation-checked);
    • otherwise the reservation is kept, and its proof re-runs on each later write.

clipboard: "replace"

  • A claimed write that never reads or keeps the prior clipboard. Clipboard read permission isn't needed.
  • No paste reservation is taken. Admission still refuses while another unresolved paste holds the gate.
  • The same exact-target focus checks apply. When the text can't be read, only the exact focus is revalidated.
  • clipboard_cleanup: "left_replaced". The result is completed only if the expected edit is observed; otherwise it is unknown with INSERTION_UNVERIFIED. The hint says to inspect before the next clipboard write, because a pending Command-V pastes whatever the clipboard holds then.

Native (peekaboo-attachment-insert.patch, appended last)

  • insertLiteralText becomes one caller of pasteThroughClipboard(write, expectation, replace, …). Text insertion behaves exactly as before; its predicate is the attachments: 0 case.
  • New attachmentInsert Bridge op (131 ops). It reuses the literal-insert response and descriptor.
  • The snapshot, lease and receipt code moves into a shared handleGuardedPaste.
  • ClipboardWriteRequest(fileURLs:) writes one public.file-url item per path through the claimed temporary write.
  • ClipboardService.prepareReplacingWrite() is added.
  • The native client gets the new fields, the allowlist entry, and the PASTE_UNVERIFIABLE_RECEIVER mapping. Its action input limit rises to 14 MiB, for image insertions only.

Docs: the tool description, the docs/tools.md row and paragraph (which links the Finder copy recipe), and the patches README entry.

Checks

  • npm run check and npm test pass (56 tests, 4 new).
  • The native build passes.
  • Swift tests, foreground and isolated:
    • ClipboardAttachmentWriteTests: 3 new;
    • ClipboardTemporaryWriteTests, ClipboardWriteRequestTests, ClipboardReadAccessTests and ClipboardPasteTransactionGateTests;
    • PeekabooBridgeRequestPlanTests, PeekabooBridgeOperationSemanticPlanTests and PeekabooBridgeOperationResultSemanticsTests, with the counts at 131.
  • Pending (screen locked): every live check, which also needs a dev host. Listed in Run the live checks deferred while the screen was locked #42.

Follow-up (not in this PR): pasting files into a Finder folder, with a host-side filesystem predicate. Until then, use desktop_clipboard_write with files, then Command-V.

…sert, or leave the clipboard replaced on request (#3)
@iamnbutler
iamnbutler force-pushed the feat/insert-attachments branch from 80194e3 to 686170a Compare October 17, 2026 08:48
@iamnbutler
iamnbutler merged commit 79f3efc into main Oct 17, 2026
2 checks passed
@iamnbutler
iamnbutler deleted the feat/insert-attachments branch October 17, 2026 08:55
iamnbutler added a commit that referenced this pull request Oct 20, 2026
… (#167)

* native2: background paste preparation, and file-dialog path entry

Pastes into an inactive app first make its window key with a SkyLight activation
record and a routed title-bar click, as Peekaboo does; desktop_dialog_answer path
enters a folder and name in an Open or Save panel.

Why: native2/Sources/DesktopMac/BackgroundPreparation.swift | TextEdit and Finder drop a background Command-V unless the window is key; #99 passed earlier only because TextEdit happened to be frontmost
Why: native2/Sources/DesktopMac/Dialogs.swift | a sheet (and Go to Folder inside it) holds keyboard focus itself, so the focused window can be a sheet inside the dialog's window

* lane: sync finder-paste memory

* lane: confirm the dialog prefix note
iamnbutler added a commit that referenced this pull request Oct 20, 2026
)

* native2: select web text by text markers; refuse an unverifiable paste receiver before checking focus

Why: native2/Sources/DesktopMac/WebText.swift | Safari on macOS 27 dropped AXStart/EndTextMarkerForTextMarkerRange; bounds(of:) reads both ends from the AXTextMarkerRange itself
Why: native2/Sources/DesktopMac/Paste.swift | the receiver check comes before requireKeyboardFocus so a background web field gets PASTE_UNVERIFIABLE_RECEIVER, as on Node, not a focus refusal

* lane: sync web-select memory

* lane: notes
@iamnbutler iamnbutler mentioned this pull request Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant