Documents the 5-phase diagnosing methodology I used for the MMA
concurrent tracks tests, adapted for the RAG test failure.
Contents:
- Part 1: What Happened (the RAG investigation summary)
- Part 2: The 5-Phase Diagnosing Methodology (code reading, file-based
logging, minimal reproduction, id() logging, fix+verify)
- Part 3: Adapted Playbook for the RAG Test (concrete steps)
- Part 4: Key Files to Investigate
- Part 5: Quick Reference Commands
- Part 6: Anti-Patterns to Avoid
- Part 7: What I'd Do Differently Next Time
- Part 8: Summary for the Future Agent (what I know, what I tried,
what I didn't try, best guess for the fix)
- Part 9: Files Created This Session
Key insight: the live_gui subprocess (session-scoped fixture) holds
file locks on the chroma collection directory. No cleanup can
remove files that the running process has open. A complete fix
requires either changing the fixture scope, using a per-test
workspace for RAG tests, or implementing a more sophisticated
lock-handling strategy in the RAG engine.
This playbook is designed to be followed by an agent after a context
compaction, with enough context to pick up where the investigation
left off.
ed
d26a2f9fce