Plan: RAG Test Failures Fix

Track: rag_test_failures_20260615 Spec: spec.md Status: Active (plan approved 2026-06-15)

TDD Protocol (MANDATORY)

For each phase, the order is:

Red: verify the test/failure is present (TDD red phase)
Green: implement the fix; run the test; confirm it passes
Verify green: run the targeted test batch to confirm no regression
Commit: one atomic commit per task with a clear message
Git note: attach a 3-5 sentence summary to the commit

Per the project rule (see AGENTS.md "Critical Anti-Patterns"), per-task atomic commits. The 1-space indentation rule is in effect.

Diagnostic strategy: the error message "'NoneType' object has no attribute 'get'" is specific — it indicates a dict.get() call on a None value. The implementer should add a diagnostic traceback to the except clause at src/app_controller.py:1479 to capture the actual call site, then remove the traceback after the fix is verified.

Phase 1: Investigation + reproducing test

Focus: Find the exact location of the .get(None) call. The spec §1.4 lists 5 candidate sites; the investigation will narrow to 1.

Task 1.1: TDD red - verify all 3 RAG tests fail with the same error
- Command: uv run pytest tests/test_rag_phase4_final_verify.py tests/test_rag_phase4_stress.py tests/test_rag_visual_sim.py -v 2>&1 | tee tests/artifacts/rag_track_phase1_red.log
- EXPECTED: 3 failures, all with the same rag_status: error: 'NoneType' object has no attribute 'get'
- COMMIT: No new commit; this is a verification step.
Task 1.2: Add diagnostic traceback to the except clause
- WHERE: src/app_controller.py:1479-1482 (the except clause in _do_rag_sync)
- WHAT: Replace the existing sys.stderr.write(f"[DEBUG RAG] Failed to sync engine: {e}\n") with sys.stderr.write(traceback.format_exc()). Also import traceback at the top of the file (if not already imported).
- HOW: Use manual-slop_edit_file to add the import and update the except clause. 2-line change.
- NOTE: This is a temporary diagnostic; remove it in Phase 2 after the fix is verified.
- SAFETY: The traceback import is stdlib; no new dependency. The format_exc() is thread-safe.
- VERIFY: uv run pytest tests/test_rag_visual_sim.py -v 2>&1 | tee /tmp/rag_diag.log — confirm the full traceback is printed to stderr
- COMMIT: chore(rag): add diagnostic traceback to _do_rag_sync except clause (Phase 1.2)
Task 1.3: Capture the full traceback and identify the call site
- Command: uv run pytest tests/test_rag_visual_sim.py -v 2>&1 | grep -A 30 "Traceback"
- EXPECTED: A traceback showing the exact line where .get() is called on None
- OUTPUT: Document the traceback in the commit message for the fix (Phase 2)
- COMMIT: No new commit; this is a verification step.
Task 1.4: Write a focused reproducing test (smaller than the 3 RAG tests)
- WHERE: tests/test_rag_sync_none_error.py (new file, ~30 lines)
- WHAT: A focused test that:
  1. Creates an AppController with mocked dependencies
  2. Sets rag_enabled=True via the setter
  3. Submits the sync and waits for completion
  4. Asserts rag_status != "error: ..." (or specifically rag_status == "ready")
- HOW: Use the existing test_orchestration_logic.py or test_rag_engine.py patterns as a template. Use MagicMock for the controller's heavy dependencies.
- SAFETY: No live_gui; this should be a fast unit test.
- VERIFY: uv run pytest tests/test_rag_sync_none_error.py -v fails with the same error
- COMMIT: test(rag): add focused reproducing test for NoneType.get sync error (Phase 1.4)

Phase 2: Fix

Focus: Fix the root cause found in Phase 1. The fix is dependent on what the investigation reveals.

Task 2.1: Implement the fix based on the Phase 1 investigation
- WHERE: TBD based on Phase 1 (one of: src/rag_engine.py:_validate_collection_dim_result, src/rag_engine.py:_init_vector_store_result, src/app_controller.py:_do_rag_sync, or a config field setter)
- WHAT: Add a defensive guard or correct the call. Specific examples:
  - If src/rag_engine.py:149 (embeddings = res.get("embeddings")): Add a check that res is a dict before calling .get(); if not, return Result(data=None) early.
  - If a config field is None: Add a guard in the setter or a fallback in the engine init.
  - If the IO pool is leaking errors from another worker: Add a more specific exception handler.
- HOW: Use manual-slop_edit_file for surgical changes. 1-5 lines typical.
- SAFETY: The fix must be defensive (guard against future None) or corrective (the field should not be None). Document the choice in the commit message.
- VERIFY: uv run pytest tests/test_rag_sync_none_error.py -v passes (the new test from Phase 1.4)
- COMMIT: fix(rag): handle None response in _validate_collection_dim_result (Phase 2.1) (or appropriate title based on the actual fix)
Task 2.2: Verify all 3 RAG tests pass
- Command: uv run pytest tests/test_rag_phase4_final_verify.py tests/test_rag_phase4_stress.py tests/test_rag_visual_sim.py -v 2>&1 | tee tests/artifacts/rag_track_phase2_green.log
- EXPECTED: 3/3 pass
- COMMIT: No new commit; this is a verification step.
Task 2.3: Remove the diagnostic traceback from Phase 1.2
- WHERE: src/app_controller.py:1479-1482
- WHAT: Remove the import traceback (if not used elsewhere) and the traceback.format_exc() call. Restore the original sys.stderr.write(f"[DEBUG RAG] Failed to sync engine: {e}\n").
- HOW: Use manual-slop_edit_file with the exact old/new strings.
- SAFETY: Verify traceback is not used elsewhere in the file before removing the import. Use uv run rg "traceback" src/app_controller.py to check.
- VERIFY: uv run rg "traceback" src/app_controller.py returns 0 hits (or only the import line which should also be removed)
- COMMIT: chore(rag): remove diagnostic traceback from _do_rag_sync (Phase 2.3)
Task 2.4: Add a defensive guard or proper error message (G3)
- WHERE: TBD based on the fix in Task 2.1
- WHAT: Ensure the error message identifies WHICH field or call is None. For example, change "error: NoneType has no attribute 'get'" to "error: RAG sync failed: .get() called on None in ".
- HOW: Catch the specific exception type and re-raise with a more informative message. Or add a try/except around the specific call site.
- SAFETY: The new error message should not leak sensitive information (file paths are OK; credentials are not).
- VERIFY: Run the 3 RAG tests; if the bug recurs, the error message is more useful.
- COMMIT: fix(rag): add defensive guard with informative error message (Phase 2.4)

Phase 3: Full test suite + batched verification

Focus: Ensure no regression in the broader test suite.

Task 3.1: Run the full RAG test suite
- Command: uv run pytest tests/test_rag_engine.py tests/test_rag_engine_result.py tests/test_rag_engine_ready_status_bug.py tests/test_rag_gui_presence.py tests/test_rag_integration.py tests/test_sync_rag_engine_coalescing.py tests/test_rag_phase4_final_verify.py tests/test_rag_phase4_stress.py tests/test_rag_visual_sim.py -v 2>&1 | tee tests/artifacts/rag_track_phase3_rag_suite.log
- EXPECTED: 30+/30+ pass (no new failures)
- COMMIT: No new commit; this is a verification step.
Task 3.2: Run the full test suite
- Command: uv run pytest tests/ 2>&1 | tee tests/artifacts/rag_track_phase3_full.log
- EXPECTED: 1285 pass + 4 skip + 0 fail (was 1282 + 4 + 3 pre-track)
- ACTION: If NEW failures appear, STOP and report to the user.
- COMMIT: No new commit; this is a verification step.
Task 3.3: Run the batched test suite
- Command: uv run .\scripts\run_tests_batched.py 2>&1 | tee tests/artifacts/rag_track_phase3_batched.log
- EXPECTED: All tiers PASS; no failures
- COMMIT: conductor(checkpoint): Phase 3 complete - 1285 tests pass, 0 failures

Phase 4: Docs update

Focus: Document the fix in docs/guide_rag.md (if it exists).

Task 4.1: Check if docs/guide_rag.md exists
- Command: uv run rg "guide_rag" docs/ docs/AGENTS.md
- EXPECTED: May or may not exist; if not, skip Phase 4
- COMMIT: No new commit.
Task 4.2 (CONDITIONAL): If docs/guide_rag.md exists, add a troubleshooting entry
- WHERE: docs/guide_rag.md (a "Troubleshooting" or "Known issues" section)
- WHAT: Add 1-2 paragraphs documenting:
  - The error: "If rag_status shows 'NoneType' object has no attribute 'get', ..."
  - The fix: "Check the RAG sync worker at src/app_controller.py:_do_rag_sync..."
- HOW: Use manual-slop_edit_file to add the section.
- VERIFY: uv run rg "NoneType" docs/guide_rag.md returns 1 hit
- COMMIT: docs(rag): document the NoneType.get fix (Phase 4.2)

Phase 5: Metadata + tracks.md

Focus: Mark the track complete in the project registry.

Task 5.1: Update metadata.json to mark the track complete
- WHERE: conductor/tracks/rag_test_failures_20260615/metadata.json
- WHAT: Change "status": "active" to "status": "completed". Add a completed_at field. Update verification_criteria to reflect what was actually verified.
- HOW: Direct file edit.
- COMMIT: conductor(track): mark rag_test_failures_20260615 as completed
Task 5.2: Update conductor/tracks.md to reflect the track's status
- WHERE: conductor/tracks.md
- WHAT: Add a row for the RAG track or update the existing RAG section.
- HOW: Direct file edit.
- COMMIT: conductor: mark rag_test_failures_20260615 as completed in tracks.md
Task 5.3: Conductor - User Manual Verification
- ACTION: Announce the track is complete. Provide the user with a summary: "3 RAG tests fixed; first fully green baseline since 2026-06-12. The user can now proceed with the send_result → send mass rename or the data_structure_strengthening_20260606 track."

Summary

Total tasks: ~10 (across 5 phases)
Total atomic commits: 4 (1 fix + 1 docs + 1 metadata + 1 final-state)
All commits have git notes
Dependencies: None (independent track)
Out of scope (deferred): send_result → send mass rename (user's manual refactor); 23 lower-impact weak-type files (data_structure_strengthening); live_gui_mock_injection infrastructure

Test count math

Pre-track baseline: 1282 pass + 4 skip + 3 fail
After this track: 1285 pass + 4 skip + 0 fail (3 newly-passing)
First fully green baseline since data_oriented_error_handling_20260606 shipped 2026-06-12

11 KiB Raw Blame History