feat: semantic search in command palette and assistant

docs/website/documentation/ai.md

@@ -69,6 +69,8 @@ Workspace selection is captured when you send each message. Use it for requests
 
 Attachments are optional and do not restrict vault search. Ask the assistant to find records across Code, Notes and HTTP, then click a record link in the answer or **Found records** to open it.
 
+The assistant uses the same search as the Command Palette: it matches words in record names and text, including HTTP methods, paths and descriptions. With [semantic search](/documentation/search#search-by-meaning) turned on, it also finds records related by meaning, for example a note about invoices when you ask about billing. Records whose name matches the query exactly rank first.
+
 ## Apply, Preview and Undo Changes
 
 The assistant can create and edit records, duplicate them, change tags and folders, move records to Trash and restore them. It can also update Code fragments and Notes [task properties](/documentation/notes/tasks), such as status, due date and priority.
@@ -146,6 +148,8 @@ The configured provider receives your messages, included history, custom instruc
 
 API keys are stored with operating-system encryption outside your vault and are not included in vault sync. If secure storage is unavailable, you can still use a connection that does not require a key.
 
+[Semantic search](/documentation/search#search-by-meaning) runs locally with the built-in model. With a local server, indexing sends the text of every indexed record and search queries to that embedding server.
+
 For local processing, use a model running on your computer. A local server address alone does not guarantee this: Ollama can also serve cloud-backed models. massCode does not automatically switch to a cloud provider when a local server is unavailable.
 
 [MCP](/documentation/mcp) connects external AI clients to your vault. It is separate from the built-in assistant and does not need to be enabled to use it.

docs/website/documentation/command-palette.md

@@ -13,7 +13,7 @@ Open it with <kbd>Cmd+P</kbd> on macOS or <kbd>Ctrl+P</kbd> on Windows and Linux
 
 ## What You Can Find
 
-The palette searches by title across searchable content:
+The palette searches across searchable content:
 
 - snippets in Code
 - notes in Notes
@@ -22,10 +22,14 @@ The palette searches by title across searchable content:
 - commands
 - recently opened items
 
-Search results are ranked with your recent usage, so items you open often move higher over time.
+Snippets, notes and HTTP requests are found by words in their names and text. Each result shows why it was found, such as **Matches the name** or **Has your words**, and can show a short excerpt from the record; words that matched are highlighted. Weaker matches appear under **Possibly related**; after the first three, they are collapsed into a single row that you can expand.
+
+Among results with similar relevance, items you open often move higher over time.
 
 Palette search also supports space scopes. In massCode 5.6 and later, it can also use tag and folder filters. See [Search](/documentation/search#search-from-the-command-palette) for the full search workflow.
 
+With semantic search turned on, the palette also finds records related to your query by meaning. Run **Turn on semantic search** to set it up. See [Search by meaning](/documentation/search#search-by-meaning).
+
 ## Run Commands
 
 Commands create new content or open app-level actions without switching spaces first.
@@ -53,14 +57,14 @@ Press <kbd>Esc</kbd> to leave command mode and return to the full palette.
 
 ## Create From Search
 
-If a search has no exact match, the palette can create a new item from the query.
+When you search for content, the palette also offers to create a new item from the query. The create actions are pinned at the bottom of the palette, above the footer.
 
 - Create a snippet in Code Inbox.
 - Create a note in Notes Inbox.
 - Create an HTTP request in HTTP Inbox.
 - Create an HTTP request from a URL.
 
-When create fallbacks are shown, press <kbd>Shift+Enter</kbd> to run the first create action.
+Press <kbd>Shift+Enter</kbd> to run the first create action without moving the selection to it. <kbd>Enter</kbd> runs a create action only when it is selected and the search for the current query has finished.
 
 ## Actions Panel
 

docs/website/documentation/search.md

@@ -1,6 +1,6 @@
 ---
 title: Search
-description: "Search snippets, notes, HTTP requests, spaces, and commands in massCode with scoped list search and Command Palette filters."
+description: "Search snippets, notes, HTTP requests, spaces, and commands in massCode with scoped list search, Command Palette filters, and search by meaning."
 ---
 
 # Search
@@ -44,7 +44,17 @@ The palette can search:
 - commands
 - recently opened items
 
-Results are ranked with recent usage, so items you open often move higher over time.
+Snippets, notes and HTTP requests are found by the words in their names and text:
+
+- snippets: name, description, fragment labels and fragment content
+- notes: name and content
+- HTTP requests: name, method, URL path and description; the host, headers, query parameters and body are not searched
+
+Words match whole words and their simple forms, so `test` also finds `tests` and `testing`. The last word you type also matches the beginning of longer words, so results appear while you are still typing. Part of a word inside a longer identifier, such as `auth` in `useAuthStore`, matches only in the record name.
+
+Among results with similar relevance, items you open often move higher over time.
+
+With [search by meaning](#search-by-meaning) turned on, the palette also finds records related to your query by meaning.
 
 ## Scope the palette to a space
 
@@ -107,3 +117,65 @@ Notes /Development #backend migration
 ```
 
 This searches Notes for `migration` inside the Development folder with the backend tag.
+
+## Search by meaning
+
+Search by words finds records that contain what you typed. With semantic search turned on, the palette also finds records related by meaning, for example a note about invoices when you search for `customer billing`. Results by words appear as you type. After a short pause, results by meaning are added below them; results already shown by words keep their order. While this happens, the footer shows **Searching by meaning...**.
+
+Each result shows why it was found:
+
+| Label | Meaning |
+| --- | --- |
+| **Matches the name** | The record name matches your query. |
+| **Has your words** | The record text contains your words. |
+| **By meaning** | The record is related to your query by meaning. |
+| **Words and meaning** | The record matches both by words and by meaning. |
+
+Under the title, a result shows a short excerpt from the record; words that matched are highlighted. A result found only by meaning shows the start of the matching part of the record without highlighting. A result that matches only by name has no excerpt.
+
+Strong matches appear under **Results**. Weaker matches appear below them under **Possibly related**: the first three are shown, and the rest are collapsed into a **N more weaker matches** row. Select it and press <kbd>Enter</kbd>, or click it, to show them. The footer shows how many results and weaker matches were found.
+
+Press <kbd>Enter</kbd> to open the selected result, including one under **Possibly related**. The first result is selected by default. While the palette is still searching for a changed query, <kbd>Enter</kbd> does not open results of the previous query and does not create a new item. Press <kbd>Shift+Enter</kbd> to create an item from the query instead. See [Create From Search](/documentation/command-palette#create-from-search).
+
+If nothing matches, the palette suggests trying words you remember from the record or describing it in your own words. The create actions from [Create From Search](/documentation/command-palette#create-from-search) are still available.
+
+Space scopes such as `@notes`, tag filters and folder filters work with both kinds of search.
+
+The [AI assistant](/documentation/ai#choose-context-or-search-the-vault) uses the same search when it looks for records in the vault.
+
+### Turn on semantic search
+
+Run **Turn on semantic search** from the Command Palette. It downloads the built-in model and opens **Preferences → Semantic Search**, where you can follow the download and indexing progress. On Intel Macs, it opens the settings so you can set up a local server.
+
+To turn semantic search off, set **Source** to **Off** in **Preferences → Semantic Search**. The palette and the assistant keep searching by words.
+
+You can also open **Preferences → Semantic Search** and choose a **Source**:
+
+| Source | How it works |
+| --- | --- |
+| **Built-in model** | Click **Download model (~210 MB)**. The model is stored in the app data folder on this computer, outside your vault, and runs offline after download. Use **Remove model** to delete it and its index. |
+| **Local server** | Uses your own Ollama or LM Studio server, separate from the AI assistant connection. Choose the **Server**, check the **Server URL**, install an embedding model on the server, for example `ollama pull embeddinggemma`, click **Refresh models**, then choose it in **Embedding model**. You can also enter the model ID manually. |
+
+After you choose a source, massCode indexes your snippets, notes and HTTP requests in the background and shows progress under **Index**. Long records are split into chunks, so progress is counted in chunks, for example **Indexing 120 of 800 chunks**. When indexing finishes, **Index** shows the number of indexed records. The built-in model indexes on two threads to keep the computer responsive, and its indexing also starts shortly after the app launches. With a local server, indexing starts after you choose an embedding model, and continues when you search from the palette or the assistant if the index is not ready yet. For HTTP requests, only the name, method, URL path and description are indexed. Changed records are reindexed in the background when you search.
+
+While the built-in model downloads or the index is being built, an icon with the progress in percent appears at the bottom of the space rail. Hover it to see the progress; search by words already works at this time. The icon turns red if the download or indexing fails. Click it to open **Preferences → Semantic Search**. The icon is not shown for short updates that finish within a second or when everything is ready.
+
+Until the first index is ready, the palette finds records by words only and the footer shows **Semantic search is warming up...**.
+
+The index is stored only on this computer and is not synced with your vault. Switching the source, server or embedding model rebuilds it. Switching to another vault also builds a new index: the built-in model starts indexing the new vault right away, and with a local server indexing starts with the first search. Moving the vault to another folder keeps the existing index.
+
+The built-in model is not available on Intel Macs; use a local server instead.
+
+::: info Privacy
+The built-in model works offline after download. With a local server, indexing sends the text of every indexed record to that server, and palette and assistant searches send your queries to it.
+:::
+
+::: info Windows requirement
+The built-in model needs the Microsoft Visual C++ Redistributable. If the model cannot start, install it from Microsoft and click **Download model** again; files that were already downloaded are reused.
+:::
+
+### Troubleshooting
+
+- No **By meaning** results: check **Index** in **Preferences → Semantic Search**. If indexing failed, the error is shown there and the palette keeps searching by words.
+- With a local server, start the server and choose an embedding model, not a chat model.
+- If the built-in model fails to download, check your connection and try again.

electron-builder.json

@@ -9,9 +9,16 @@
     "build/renderer/**/*",
     "build/main/**/*",
     "build/shared/**/*",
-    "!build/**/*.map"
+    "!build/**/*.map",
+    "!**/node_modules/onnxruntime-web/**",
+    "!**/node_modules/onnxruntime-node/bin/napi-v6/darwin/arm64/libonnxruntime.1.30.0.dylib"
+  ],
+  "asarUnpack": [
+    "node_modules/@lydell/**/*",
+    "**/node_modules/onnxruntime-node/**/*",
+    "**/node_modules/sharp/**/*",
+    "**/node_modules/@img/**/*"
   ],
-  "asarUnpack": ["node_modules/@lydell/**/*"],
   "publish": {
     "provider": "github",
     "owner": "massCodeIO",
@@ -23,15 +30,24 @@
     "entitlements": "build/entitlements.mac.inherit.plist",
     "category": "public.app-category.productivity",
     "hardenedRuntime": true,
-    "notarize": true
+    "notarize": true,
+    "files": ["!**/node_modules/onnxruntime-node/bin/napi-v6/{linux,win32}/**"]
   },
   "win": {
     "target": ["nsis", "portable"],
-    "icon": "build/icons/icon.ico"
+    "icon": "build/icons/icon.ico",
+    "files": [
+      "!**/node_modules/onnxruntime-node/bin/napi-v6/{darwin,linux}/**",
+      "!**/node_modules/onnxruntime-node/bin/napi-v6/win32/arm64/**"
+    ]
   },
   "linux": {
     "target": ["AppImage"],
-    "icon": "build/icons/icon.png"
+    "icon": "build/icons/icon.png",
+    "files": [
+      "!**/node_modules/onnxruntime-node/bin/napi-v6/{darwin,win32}/**",
+      "!**/node_modules/onnxruntime-node/bin/napi-v6/linux/arm64/**"
+    ]
   },
   "nsis": {
     "oneClick": false,

package.json

@@ -79,6 +79,7 @@
     "@elysiajs/swagger": "^1.3.1",
     "@excalidraw/excalidraw": "^0.18.1",
     "@faker-js/faker": "^10.1.0",
+    "@huggingface/transformers": "4.3.1",
     "@internationalized/date": "3.7.0",
     "@lezer/highlight": "^1.2.3",
     "@lezer/markdown": "^1.6.3",
@@ -199,13 +200,24 @@
     "wait-on": "^9.0.4"
   },
   "pnpm": {
+    "ignoredBuiltDependencies": [
+      "onnxruntime-node",
+      "protobufjs"
+    ],
     "onlyBuiltDependencies": [
       "@parcel/watcher",
       "electron",
       "esbuild",
       "simple-git-hooks",
       "vue-demi"
     ],
+    "packageExtensions": {
+      "@huggingface/transformers@4.3.1": {
+        "dependencies": {
+          "onnxruntime-common": "1.30.0"
+        }
+      }
+    },
     "patchedDependencies": {
       "httpsnippet@3.0.10": "patches/httpsnippet@3.0.10.patch"
     }