Skip to content

LangChain: agnostic spec on guide 5caba53 (old-guide baseline) - #74

Open
Carr1005 wants to merge 1 commit into
course/langchainfrom
spec/langchain-5caba53
Open

Carr1005 wants to merge 1 commit into
course/langchainfrom
spec/langchain-5caba53

Conversation

@Carr1005

Copy link
Copy Markdown
Collaborator

Adds courses/langchain/spec.md, generated on guide 5caba53. It stacks on #73 (materials), so merge #73 first.

Purpose: this is the old-guide baseline. The guide is being redesigned (decisions agreed 2026-10-23, kept untracked in experiments/guide-redesign/decisions.md). This spec is what the new guide's LangChain spec will be compared against, by building from both and by the owner reading both specs' questions. It is not presented as the target design.

How it was made (Opus 5.5)

  1. Clean-room generation from the guide and courses/langchain/materials/ only.
    • Result: a catalog Q&A assistant with conversation memory and self-evaluation, 14 Ledger rows, 13 rules, 21 tests (18 offline, 3 live). Checker: 0 findings.
    • Document splitting is a row (D1), reached by the generator itself.
  2. Audit round 1: 12 hard, 11 soft.
    • 11 consistency items were repaired. These are named in §7 option lines, plus --offline on offline tests, two test Givens, and CTX-C definitions.
    • No row, option, default, rule, test outcome or fixture was changed (owner rule U10).
    • The diff is 17 lines. The provenance line says "Hand-evolved since 2026-10-22".
  3. Audit round 2: 9 hard, 10 soft. Nothing repaired; stopped by owner decision.

Open points (audit round 2)

Hard, need a decision (not repaired):

  1. D3e and D3f (vector and entity memory) break D3's own must-hold rule ("oldest dropped first").
  2. RR1b (OpenAI embeddings) may break RR1's rule "same text → same vector".
  3. D3d's limit-20 test: it is unstated whether the summary counts toward the limit.
  4. L3b needs a scripted listing answer that §5 does not declare.
  5. D6d ("final answer only") is offered while CTX-B6 lists judging by the final answer as a failure.
  6. T12 (one-line index vs step-by-step) is filed as narrative although both were shown working.
  7. T14 (in-memory vs persistent vector store) is structural but filed as narrative.

Hard, consistency (not repaired):

  • RR1a's cell still says "in trace output", which D6d removes.
  • AC18's secret search (sk- + letters) matches "Ask-First" inside the spec, which /prepare-build copies into the build folder.

Soft: 10, mostly narrative used without a never-shown reason (T3, T10, T11, T21–T24), undefined terms, two schema/AC naming gaps, and the window option listed alongside a "window forgets" failure.

Several of these are guide-level patterns the redesign addresses: options that break their own row's rule, and the two conflicting narrative sentences in the guide.

🤖 Generated with Claude Code

Clean-room generation (Opus 5.5) plus 11 consistency-only repairs from
audit round 1. Opened as the old-guide baseline the redesigned guide
will be benchmarked against.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant