Private
Public Access
0
0
Files
manual_slop/docs/guide_agent_memory_dimensions.md
T
ed 434b6d0d54 docs: reduce redundant content across files; map references to canonical sources
Per user 'a bunch of docs just committed had redundant content across
files. Can we do a reduction of that and instead map references to
other files?'

This commit reduces content duplication across 9 files. The
canonical sources are kept as detailed references; the other
files now point to them.

Reductions (table replaced with 'see canonical' reference):

1. data_oriented_design.md §9: the 4-dim memory table
   (canonical: conductor/code_styleguides/agent_memory_dimensions.md §0)

2. guide_agent_memory_dimensions.md §0: the 4-dim memory table
   (canonical: conductor/code_styleguides/agent_memory_dimensions.md §0)

3. guide_caching_strategy.md §1: the 12-layer model
   (canonical: conductor/code_styleguides/cache_friendly_context.md §1)

4. guide_ai_client.md 'Cache strategy' section: the 12-layer model recap
   (canonical: conductor/code_styleguides/cache_friendly_context.md §1)

5. guide_knowledge_curation.md §1: the 5 category file details
   (canonical: conductor/code_styleguides/knowledge_artifacts.md §1)

6. product-guidelines.md 'Memory Dimensions' section: the 4-dim table
   (canonical: conductor/code_styleguides/agent_memory_dimensions.md §0)

7. guide_mma.md '4 memory dimensions' section: the MMA scope table
   (canonical: conductor/code_styleguides/agent_memory_dimensions.md §0)

8. docs/AGENTS.md §0 + §5-§8: 4-dim table + caching/knowledge/RAG/
   feature flag tables (canonical: the per-topic styleguides in
   conductor/code_styleguides/)

9. AGENTS.md 'Code Styleguides' section: the 6-styleguide list
   (canonical: docs/AGENTS.md §2)

The principle: each piece of content has ONE source of truth; other
places point to it. The data-oriented way. Files retain their
narrative flow and the 'what this is' intros, but the detailed
tables are now in their canonical home.

Net effect: -2100 bytes across 9 files (without losing any
information - the canonical sources are unchanged). The
'cross-references' sections are kept; the duplicated content
is removed.
2026-06-12 14:10:30 -04:00

14 KiB

The 4 Memory Dimensions (cross-cutting guide)

Status: User-facing cross-cutting guide on the 4 memory dimensions. For agents, see ./docs/AGENTS.md §0. Date: 2026-06-12 Cross-refs: conductor/code_styleguides/agent_memory_dimensions.md; docs/guide_context_curation.md; docs/guide_rag.md; docs/guide_knowledge_curation.md; conductor/tracks/nagent_review_20260608/nagent_review_v2_3_20260612.md §2.8.

What this is. The conversation data has 4 distinct memory dimensions. Most features touch 1-2; some touch 3. This guide is the cross-cutting reference: when to use which dimension, the boundaries between them, and the decision tree for "which dim does this feature need?"


0. The 30-second version

Manual Slop has 4 memory dimensions for the conversation data. The full canonical table is in conductor/code_styleguides/agent_memory_dimensions.md §0 (curation / discussion / RAG / knowledge, with the SSDL shape tag per dim).

The one-line summary: curation is per-file structural; discussion is per-turn conversational; RAG is opt-in semantic; knowledge is per-project durable. Pick the matching dimension; don't reach for the wrong shape.

The decision tree:

Q: What is the *data* the feature needs?
   │
   ├── "How to render a file"          ──► Curation (FileItem)
   ├── "What was said in this chat"     ──► Discussion (disc_entries)
   ├── "What similar content exists"    ──► RAG (RAGEngine.search)  [opt-in]
   └── "What we learned from past runs" ──► Knowledge (knowledge/digest.md)

Pick the matching dimension. If the feature needs 2+, use 2+ — but be explicit about which is primary and which is secondary.


1. Curation memory (per-file, per-discussion, structural)

The shape. Per-file curation config in FileItem:

  • path (the file identity)
  • auto_aggregate (include in auto-aggregation?)
  • force_full (bypass aggregation with full content?)
  • view_mode (full / skeleton / summary / sig / def / agg)
  • ast_signatures (signatures only?)
  • ast_definitions (definitions only?)
  • ast_mask (per-symbol mask)
  • custom_slices (Fuzzy Anchors)

A ContextPreset is a named, persisted set of FileItems. Both persist in the project TOML.

The query model. "When discussion X opens, render file Y per its curation memory." Implicit in aggregate.py:run at discussion start. The user doesn't query the curation memory directly; they configure it.

The right tool. The Structural File Editor (per docs/guide_context_curation.md). AST-aware slices, Fuzzy Anchor slices, view-mode picker. The file's FileItem is the UI surface.

The wrong tool. Storing curation state in disc_entries (it's not conversational). Storing curation state in the RAG index (it's structural, not semantic). Storing curation state in the knowledge digest (it's per-discussion, not durable).

The codepath (SSDL):

[Q:discussion starts]
   │
   ▼
[Q:which ContextPreset is active?]
   │
   ├── preset N ──► [I:load ContextPreset N's FileItems]
   │
   ▼
[loop: each FileItem]
   │
   ├──► [Q:FileItem.view_mode?]
   │     ├── full ──► [I:read full file]
   │     ├── skeleton ──► [I:py_get_skeleton / ts_c_get_skeleton]
   │     ├── summary ──► [I:run_subagent_summarization]
   │     ├── sig ──► [I:py_get_skeleton (signatures only)]
   │     ├── def ──► [I:py_get_skeleton (definitions only)]
   │     └── agg ──► [I:py_get_skeleton (children only)]
   │
   ├──► [Q:FileItem.ast_mask?] ──► [I:apply ast_mask to the rendered view]
   ├──► [Q:FileItem.custom_slices?] ──► [I:apply custom_slices]
   └──► [I:append to aggregate markdown]

The shape rule. Curation is per-file, per-discussion, structural. Edited at the Structural File Editor. Persisted in TOML. The file's FileItem is the single source of truth for "how do I render this file in the AI's context."

See: docs/guide_context_curation.md; src/models.py:510-559 (FileItem schema); src/context_presets.py (ContextPresetManager).


2. Discussion memory (per-discussion, conversational, multi-turn)

The shape. app.disc_entries: list[dict] where each entry is {"role": str, "content": str, "collapsed": bool, "ts": str, ...} plus optional thinking_segments and usage (token accounting). The discussion is rendered as a list[Message] for the LLM by build_markdown (per src/aggregate.py).

The query model. "What did the user say? What did the AI say? In what order?" The discussion is the prior context for the next LLM call. The user can edit, insert, delete, role-change, and branch at any entry (A1-A7 per-entry operations per the nagent review v1 §3).

The right tool. The Discussion Hub panel. Per-entry [Edit], [Read], [+/-], Ins, Del, [Branch], role combo. The undo/redo stack (UISnapshot) and the Take/branching/compact system.

The wrong tool. Storing discussion state in the RAG index (it's temporal, not semantic). Storing discussion state in the knowledge digest (it's per-discussion, not durable). Storing discussion state in a FileItem (it's not per-file).

The codepath (SSDL):

[Q:user types prompt + hits Enter]
   │
   ▼
[I:append new entry to disc_entries]    (role: "User")
   │
   ▼
[Q:which ContextPreset is active?] ──► [I:render FileItems per curation memory]
   │
   ▼
[I:aggregate.build_markdown(preset, discussion) -> str]
   │
   ▼
[I:ai_client.send(aggregate_text, history)]
   │
   ▼
[I:append new entry to disc_entries]    (role: "AI", content: response)
   │
   ▼
[Q:user pressed Edit on an entry?] ──► [I:update disc_entries[i].content]
[Q:user pressed Branch on an entry?] ──► [I:project_manager.branch_discussion(index) -> new Take]
[Q:user pressed Undo?] ──► [I:history.UISnapshot.pop() -> restore previous state]
[Q:user pressed Compact?] ──► [I:ai_client.run_discussion_compaction(discussion)]

The shape rule. Discussion is per-discussion, conversational, multi-turn. Edited per-entry. Persisted in TOML via _flush_to_project. The disc_entries list is the single source of truth for "what was said in this discussion."

See: docs/guide_architecture.md §"Threading model"; src/gui_2.py:3770-3853 (render_discussion_entry); src/history.py:8-71 (UISnapshot).


3. RAG memory (opt-in, semantic, fuzzy)

The shape. ChromaDB vector store; per-file FileItem-like records with embeddings. RAGEngine.search(query, k=N) returns the top-N most-similar chunks. Persisted in ~/.manual_slop/.slop_cache/chroma_<embedding_provider>/.

The query model. "Given a query, return similar content from the indexed corpus." Semantic similarity, fuzzy. No provenance beyond the file path. No user-editable content.

The right tool. RAGEngine.search() at LLM call time (the rag_* results injected into the LLM prompt). The [X] Enable RAG toggle in AI Settings. The RAGConfig (embedding provider, chunk size, chunk overlap, source selection).

The wrong tool. Using RAG as a replacement for the other 3 dimensions. Using RAG results for state mutation (the integration discipline prohibits this). Using RAG for "show me the last thing the user said" (use Discussion memory). Using RAG for "show me what we decided last time" (use Knowledge memory).

The codepath (SSDL):

[Q:ai_client.send() is called]
   │
   ▼
[Q:is RAG enabled?]
   │
   ├── no ──► [T:skip]
   │
   ▼
[Q:which RAG source?]
   │
   ├── project ──► [I:RAGEngine.index_file for each file in project]
   ├── global ──► [I:RAGEngine.index_file for each file in ~/.manual_slop/knowledge/]
   └── none ──► [T:skip]
   │
   ▼
[Q:RAG engine initialized?]
   │
   ├── no ──► [I:RAGEngine._init_embedding_provider()]   (lazy init, may download)
   │
   ▼
[I:RAGEngine.search(query, k=N) -> Result[list[SearchResult], ErrorInfo]]
   │
   ▼
[I:append "{rag-context}" block to aggregate markdown]

The shape rule. RAG is opt-in. Default-off. Complements the other dimensions; never replaces. Provenance is required (file path, chunk offset). No mutation.

See: docs/guide_rag.md; conductor/code_styleguides/rag_integration_discipline.md; src/rag_engine.py:1-384.


4. Knowledge memory (per-project, durable, provenance-aware)

The shape. A markdown tree at ~/.manual_slop/knowledge/:

File Format What it stores
facts.md - {statement} {provenance} Durable statements about systems, repos, tools
decisions.md - {statement, reason} {provenance} Decisions that were made
questions.md - {question} {provenance} Unanswered questions
playbooks.md - **{name}**: {steps} {provenance} Reusable command sequences
tasks.md - {task} (## Open / ## Done) Open and done tasks
files/{file_id}.md - {note} {provenance} Per-file notes (keyed by inode)
digest.md bounded 4KB The projected digest (injected as {knowledge} block)
ledger.json {entries: {sha256: {status, at, items}}} The harvest audit log

The query model. "Given past sessions, what durable knowledge should I inject into the current discussion?" The answer is the {knowledge} block in the initial context, regenerated from the category files (newest first), bounded to 4KB.

The right tool. The harvest CLI (python -m src.knowledge_harvest) for the harvest; the plain text editor for the category files. The "Knowledge" panel in the GUI for browse/edit/prune.

The wrong tool. Treating the knowledge digest as state (it's a projection; the category files are the state). Letting the digest grow unbounded (4KB cap; truncate with a visible note). Treating the per-file notes as a replacement for FileItem curation (different dimensions; both are useful).

The codepath (SSDL):

[Q:discussion starts]
   │
   ▼
[Q:knowledge digest exists?]
   │
   ├── no ──► [T:skip]
   │
   ▼
[Q:digest within 4KB budget?]
   │
   ├── yes ──► [I:read digest]
   ├── no ──► [I:read digest (truncated with note)]
   │
   ▼
[I:append "{knowledge}" block to stable prefix]   (layer 7)
   │
   ▼
[Q:per-file knowledge for files in scope?]
   │
   ├── yes ──► [I:append "{file-knowledge}" per FileItem]

The shape rule. Knowledge is per-project, durable, provenance-aware. Edited by the user (plain markdown). The category files are the source of truth; the digest is a projection. "Delete to turn off": rm digest.md → no injection.

See: docs/guide_knowledge_curation.md; conductor/code_styleguides/knowledge_artifacts.md.


5. The boundaries (when NOT to mix)

Don't store... In... Because...
Discussion state FileItem (curation) Discussion is per-discussion, not per-file
File curation disc_entries (discussion) Curation is per-file structural, not conversational
Semantic search results disc_entries (discussion) RAG is fuzzy; the discussion is precise
A long conversation the knowledge digest The digest is bounded (4KB); the conversation is unbounded
A "this is the current state" fact the RAG index RAG is semantic; state is precise
Per-file notes the discussion context The notes should follow the file, not the discussion
Per-discussion summary the knowledge digest The digest is cross-discussion, not per-discussion
LLM-derived curation the FileItem schema LLM outputs are untrusted; the FileItem is user-edited
Untrusted LLM output the knowledge category files The harvest has retry + graceful failure; but the category files are user-editable, so corrections are first-class

The discipline. When designing a new feature, ask: which of the 4 dimensions is the natural home? Don't reach for the RAG because "it's there"; reach for the dimension whose shape matches the data.


6. The decision tree (the 1-question test)

When a feature needs some memory, ask this single question:

Q: What is the *data* (not the operation) the feature needs?
   │
   ├── "How to render a file"          ──► Curation (FileItem)
   ├── "What was said in this chat"     ──► Discussion (disc_entries)
   ├── "What similar content exists"    ──► RAG (RAGEngine.search)  [opt-in]
   └── "What we learned from past runs" ──► Knowledge (knowledge/digest.md)

Pick the matching dimension. If the feature needs 2+, use 2+ — but be explicit about which is the primary (the one that holds the answer) and which is secondary (the one that provides context).


7. The cross-cutting principle (the "data is the thing")

All 4 dimensions share one principle: the data is the thing, not the agent. Each dimension has:

  • A flat shape (no object graphs; structs of structs of scalars)
  • A durable storage (TOML, ChromaDB, markdown — not Python objects)
  • A user-editable surface (the Structural File Editor, the Discussion Hub, the RAG toggle, the category files)
  • A query model that returns "data, not control flow" (per data_oriented_error_handling_20260606)

The wrong shape for the right question is a common mistake. The right question is "which of the 4 dimensions is this?" — not "is there a tool that does X?"


8. The cross-references

  • conductor/code_styleguides/agent_memory_dimensions.md — the canonical styleguide
  • docs/guide_context_curation.md — the existing curation deep-dive (dimension 1)
  • docs/guide_rag.md — the existing RAG deep-dive (dimension 3)
  • docs/guide_knowledge_curation.md — the new knowledge guide (dimension 4)
  • docs/guide_caching_strategy.md — where the 4 dims get injected in the cache strategy
  • conductor/tracks/nagent_review_20260608/nagent_review_v2_3_20260612.md §2.8 — the nagent-origin pattern that informed this guide