Skip to content

agent-memory: adopt the agnostic spec regenerated on guide 5caba53 - #63

Open
Carr1005 wants to merge 1 commit into
mainfrom
spec-agent-memory-5caba53
Open

Carr1005 wants to merge 1 commit into
mainfrom
spec-agent-memory-5caba53

Conversation

@Carr1005

Copy link
Copy Markdown
Collaborator

Replaces the agent-memory agnostic spec.md (guide bbe7189, September) with a fresh clean-room generation on guide 5caba53 (PR #62; merge #62 first so main has the guide this spec pins). This regeneration replaces the planned D4 hand edit (owner decision 2026-10-19).

How it was made (Opus 5.5)

  1. A clean-room generation from the guide and course materials only. Checker: 0 findings.
  2. Two independent audits, each followed by a repair pass. Repairs fix consistency only: counts, fixtures, citations, schemas and "Tests by option" cells. They make no decisions and add no rules or tests (owner decision on U10).
  3. A script compared the generated spec with the final one:
  4. A purpose probe on the generation took the "conversation-only" option. 44 of 56 items were covered, left out or adapted as the guide intends, 8 sat in the gate and 4 needed a guess (details in Guide: rows never restate parts another option removes; gate step 5 leaves out what exists only for a retired rule and records adapted criteria #62).

What changed for a learner

  • 14 decisions (was 13). New option ids (D1a); rows that stop mattering are skipped ("Applies only when D1 is D1a"). Labels: "(Recommended)" and "(course default)".
  • Picking a part-removing option (e.g. conversation-only) no longer leaves later questions contradicting it. The checklist records what each choice removed, and anything incompletely stated is listed as Adapted.
  • A plain-language table of every decision and option is in the first comment below, for review.

Known open points (decisions, not repaired; going to #48)

  • D4b (lossy summarization) is offered, though the spec's own failure catalog names lost detail as a failure.
  • RR2's Invariant says every search tool saves its results; the candidate-search tool saves nothing.
  • Disputed trade-off dispositions: T3 (narrative), T9 (lever with a "manual" side), T24/T25 (no tie-breaker named).
  • D2a now points to §5 for which tools are rewritten, rather than listing them at the gate.

🤖 Generated with Claude Code

Clean-room generation (Opus 5.5) on guide 5caba53 (PR #62), then two
independent audit and repair rounds. Repairs fixed consistency defects only;
a script compared the generated and final spec and found no Ledger row's
category, condition, default, option ids or labels changed and no rule or
test added or removed. Four Ledger cells were reworded only to stop naming
parts another row removes. Checker 0; gate text unchanged.

Replaces the spec pinned to guide bbe7189 (owner decision 2026-10-19,
replacing the D4 hand edit).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Carr1005

Copy link
Copy Markdown
Collaborator Author

Decision Ledger for review: agent-memory

A plain-language version of the spec's Decision Ledger, for review. Each heading links to its row in this PR's spec.md.

How to read it. Before building, the build agent asks the learner one question per decision below. Each table lists the answers they can pick.

  • ✅ Recommended: what the default build uses (chosen when the spec was written, not tailored to the learner).
  • 📘 Course used: what the course itself did.
  • If chosen: what the build leaves out or changes. "—" means nothing.

Some questions are asked only after a certain earlier answer; they say so.


L1 · What the assistant is for

The learner is asked: what are you building? Why it's a choice: the default is the course's own example; a learner with a different project swaps it.

Ref Option Setup Labels If chosen
L1a A command-line research assistant that remembers conversations across sessions and works with research papers None ✅ 📘 —
L1b An assistant for your own field, on the same memory design, with your own tools and starting data None The tools, starting data and the assistant's role line become yours; their tests are rewritten to match

L2 · What the paper library starts with

The learner is asked: what data does the assistant start with? Why it's a choice: the course loaded 100 real arXiv records; the default uses invented records so the build runs offline.

Ref Option Setup Labels If chosen
L2a Five invented paper records, loaded by a seed command None ✅ —
L2b The first 100 records of the public arXiv dataset on Hugging Face A network download, no key 📘 The loading test checks up to 100 real records and needs the network
L2c Your own documents, one record each Your files The loading test checks your records

L3 · What "working" means

The learner is asked: what should the finished assistant be able to do? Why it's a choice: the default is the course's closing demo; a learner can state their own goal.

Ref Option Setup Labels If chosen
L3a Ask a follow-up that depends on an earlier turn, restart the program, and still get answers that use earlier turns, including details recovered from summarized conversation None ✅ 📘 —
L3b A goal you state (what search must find, what answers must guarantee) None Tests for your goal are added; the rest stay

L4 · Which AI model provider, for live use

The learner is asked: which model provider should the assistant use? Why it's a choice: the course used OpenAI; another provider changes the message format, not the memory design. The offline tests never call a live model.

Ref Option Setup Labels If chosen
L4a OpenAI, with tool calling An OpenAI API key (live use only) ✅ 📘 —
L4b Anthropic, with tool use An Anthropic API key Tool messages use Anthropic's format; the key check names its key
L4c A local open-weights model behind an OpenAI-compatible server Heavy: a local model server and hardware able to run it Tool messages use the server's format; the key check becomes a server-address check

L5 · Where it runs

The learner is asked: where should the assistant run? Why it's a choice: the course ran in notebooks and wiped its tables each lesson; memory surviving restarts is the course's point, so the default keeps data between runs.

Ref Option Setup Labels If chosen
L5a A local command-line program with a database file on your machine None ✅ —
L5b Jupyter notebook cells, one call per question None stated 📘 Commands become notebook cells; the restart test uses two notebook sessions

L6 · What this build leaves out

The learner is asked: here is everything this build does not include; keep it, or bring something back? (The question shows the full list.) Why it's a choice: the course made no choice here.

Ref Option Setup Labels If chosen
L6a Keep everything left out as listed None ✅ —
L6b Bring one or more items back, named in free text Per item (each item's note in the list says) The items are built; tests are added only for items marked as restorable

D1 · Which memories the assistant keeps

The learner is asked: should the assistant keep a full set of memory types, or only the chat history? Why it's a choice: Lesson 2 shows a chat-history-only agent working, then argues it isn't enough; Lesson 3 builds the full set.

Ref Option Setup Labels If chosen
D1a Seven kinds of memory: chat history, a tool log, a paper library, past workflows, a tool catalogue, people and things mentioned, and summaries None ✅ 📘 —
D1b Chat history only; every tool is offered on every turn None shown in the course, no code shipped Saving papers, past workflows, tool search, people/things memory and summaries. 30 tests no longer apply. Skips 5 later questions (D2, D3, D4, RR2, C1)

D2 · Rewriting tool descriptions to make tools easier to find

Asked only if D1 = D1a. The learner is asked: should the model rewrite each tool's description when the tool is added? Why it's a choice: Lesson 4 shows rewritten descriptions find the right tool more reliably, at the cost of two model calls per tool; plain descriptions also work.

Ref Option Setup Labels If chosen
D2a As the course's final agent: some tools rewritten, others plain None ✅ 📘 —
D2b Every tool rewritten None One test checks that all tools are marked rewritten
D2c No tool rewritten: each tool is found by its name, its own description and its parameters None Leaves out the rewrite step; 1 test no longer applies

D3 · What runs automatically, and what the model decides to do

Asked only if D1 = D1a. The learner is asked: which memory steps run automatically every turn, and which does the model choose to call? Why it's a choice: Lesson 3 argues automatic steps are predictable and complete, while model-called steps save cost and use judgment.

Ref Option Setup Labels If chosen
D3a The course's split. Automatic: reading memory, choosing tools, shrinking the conversation when it gets too long, saving each turn. Model-called: summarize, expand a summary, search tools, paper tools None ✅ 📘 —
D3b As D3a, but saving people and things mentioned becomes a tool the model calls None never built by the course; the spec gives no tests for it Leaves out automatic saving of people and things; 3 tests no longer apply
D3c As D3a, but summaries happen only when the model asks; nothing shrinks the conversation automatically at the limit None Leaves out automatic shrinking; 1 test no longer applies

D4 · Can a shrunk conversation be recovered?

Asked only if D1 = D1a. The learner is asked: when the conversation is summarized, should the original messages stay retrievable? Why it's a choice: Lesson 5 argues a plain summary always loses detail, while compaction keeps the originals so the model can pull them back.

Ref Option Setup Labels If chosen
D4a Recoverable: a short summary with an ID replaces the conversation; the originals stay in the database, and the model can expand them None ✅ 📘 —
D4b Lossy: the summary replaces the conversation and the originals are not kept None described in the course, never built; the spec gives no tests for it Leaves out expanding a summary; 3 tests no longer apply

RR1 · Which database stores the memories

The learner is asked: which database should hold the assistant's memories? Why it's a choice: the course used Oracle, which needs an admin-installed database; the lesson is about the memory design, not the product.

Ref Option Setup Labels If chosen
RR1a A single SQLite file, built into Python None ✅ —
RR1b Oracle AI Database 26ai Free, as in the course Heavy: a local database container, an admin account, the Oracle driver 📘 The tests need the database running; two ranking tests allow ties
RR1c PostgreSQL with the pgvector extension Heavy: a database server you install and run The same as RR1b

RR2 · A web-search tool

Asked only if D1 = D1a. The learner is asked: should the assistant have a web-search tool that needs a key? Why it's a choice: Lesson 4 teaches "search and save" with Tavily, but the course's final agent doesn't use it; the paper tools already save what they find.

Ref Option Setup Labels If chosen
RR2a No web search; the paper tools save what they find None ✅ —
RR2b Tavily web search, saving each result A Tavily API key 📘 Adds the tool; its check runs live
RR2c A local fake web search, clearly labelled as fake, returning two fixed results None Adds the tool

RR3 · Which model the tests run against

The learner is asked: should the offline tests use a scripted fake model or the live one? Why it's a choice: a scripted model makes every test repeatable offline; a live model's answers vary from run to run.

Ref Option Setup Labels If chosen
RR3a A scripted fake model, defined in the spec and labelled as fake None ✅ —
RR3b The course's live OpenAI models Heavy: an OpenAI API key, paid usage on every run 📘 About 30 tests check only the shape of answers, not exact text

C1 · Do summaries belong to one conversation thread?

Asked only if D1 = D1a. The learner is asked: should a summary be visible only in the thread it came from? Why it's a choice: the course did it both ways (Lesson 5 without threads, Lesson 6 with them).

Ref Option Setup Labels If chosen
C1a Each thread sees only its own summaries None ✅ 📘 —
C1b Every thread sees all summaries (the Lesson 5 notebook's way) None Summaries record no thread; 2 tests change

Known open points (already flagged; tracked on #48)

  1. D4b (lossy) is offered as a choice, but the spec's own list of failures names "summaries lose details" as a failure. It's open whether it should be a choice at all.
  2. RR2's rule says every search tool saves its results, but the paper search tool only lists candidates and saves nothing (the fetch tool does).
  3. D2a doesn't list which tools get rewritten; it points to the spec's test section for that.

Reviewers: comment with the Ref (for example "D1b: …").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant