Phase 5 (1 of 9 cruft sites obliterated):
OBLITERATED: RAGEngine._chunk_code wrapper. It delegated to _chunk_code_result
and provided a fallback to _chunk_text on AST failure.
Migration: index_file() now calls _chunk_code_result directly with .ok check
+ chunk-size threshold check + fallback to _chunk_text inline. The structured
ErrorInfo is propagated if needed (no caller currently consumes it).
Sub-track 5 tests updated:
- tests/tier2/phase13_invariant_test.py: _chunk_code moved to obliterated list
- tests/tier2/phase13_site2_test.py: _legacy_no_broad_except -> _legacy_obliterated
- tests/test_cruft_removal.py: 2 new tests (wrapper-obliterated invariant +
caller-uses-result invariant)
PITFALL encountered: the edit_file tool removed a leading space on the
next class method's 'def' line, causing an IndentationError. Fixed by
binary-write replacement preserving CRLF + leading-space styleguide convention
(project uses 1-space indentation; class body methods start at column 1).
Test result: 124/124 pass.
Audit gate: src/rag_engine.py --strict exits 0 (no new violations).
Wrapper count: 3 -> 2 (Phase 6 remaining: gui_2 2).
Site 5 (BC at L290): _async_search_mcp (nested in _search_mcp) had:
try:
data = json.loads(res_str)
if isinstance(data, list): return data
elif isinstance(data, dict) and 'results' in data: return data['results']
return []
except:
return []
Body: bare 'except:' + return [] = empty default = SS-style violation.
Migrated to Result[T] via new module-level helper _parse_search_response_result:
- Returns Result(data=parsed_list) on success
- Returns Result(data=None, errors=[ErrorInfo]) on JSON parse failure
- Handles the list/dict/no-results branch logic
The helper is module-level (does not use self) and is placed BEFORE
class RAGEngine to avoid breaking the class definition (a def at column 0
inside a class ends the class prematurely).
Legacy _async_search_mcp delegates to the helper; on Result errors,
returns [] (preserving the original behavior).
Audit: rag_engine BC 1 -> 0; migration-target: 0.
Remaining 4 INTERNAL_RETHROW sites are Pattern 1/3 of the styleguide
(known audit limitation).
index_file had 3 try/except sites with similar patterns:
Site 3 (BC at L247): try: mtime = os.path.getmtime(full_path); except Exception: return
Site 4 (BC at L261): try: with open(full_path, ...) as f: content = f.read(); except Exception: return
Site 6 (SS at L255): try: res = self.collection.get(...); ...; except Exception: pass
Body: broad catch + early return/pass = SS-style violation.
New helpers:
- _get_file_mtime_result(full_path) -> Result[float]
Catches OSError only (specific to file stat failures).
- _check_existing_index_result(file_path, mtime) -> Result[bool]
Catches broad Exception (chromadb collection.get failures vary).
Returns data=True if already indexed (skip), data=False if needs re-indexing.
- _read_file_content_result(full_path) -> Result[str]
Catches (OSError, UnicodeDecodeError) (file I/O + encoding failures).
Legacy index_file calls each helper; on Result errors, returns early
(preserving the original behavior of skipping the file on failure).
Audit: rag_engine BC 3 -> 1 (L341 _async_search_mcp remaining).
SS: 1 -> 0.
Site 2 (BC at L224): _chunk_code had a fallback to text chunking on any
failure:
try:
parser = ASTParser('python')
tree = parser.parse(content)
...
return chunks
except Exception:
return self._chunk_text(content)
Body: broad catch + fallback to a different implementation = empty-default
fallback = SS-style violation.
New helper _chunk_code_result(content, file_path) -> Result[List[str]]:
- Returns Result(data=chunks) on AST parse success
- Returns Result(data=None, errors=[ErrorInfo]) on parse failure
Legacy _chunk_code calls helper; on Result errors, falls back to
_chunk_text (preserving original behavior). The catch logic is in the
legacy, not the helper, so the caller decides the fallback strategy.
Audit: rag_engine BC 4 -> 3.
Site 1 (BC at L33) was:
except Exception as e:
sys.stderr.write(f'FAILED to import sentence_transformers: {e}')
sys.stderr.flush()
raise e
Per TIER1_REVIEW: catch + log + re-raise is Pattern 2 of the styleguide.
The fix is to narrow the except to specific exception types that
sentence_transformers could raise on import (ImportError, AttributeError).
Refactored to:
except (ImportError, AttributeError) as e:
sys.stderr.write(f'FAILED to import sentence_transformers: {e}')
sys.stderr.flush()
raise
The bare 'raise' re-raises the current exception being handled,
preserving the original type and traceback. (Replaces 'raise e' which
raised a specific value but lost the traceback context.)
Audit: rag_engine BC 5 -> 4. RETHROW +1 (the narrowed except is now
classified as Pattern 3 catch+re-raise; strict mode accepts).
Two bugs in src/rag_engine.py were causing 'NoneType object has no attribute get'
in the live_gui RAG tests (test_rag_phase4_final_verify,
test_rag_phase4_stress):
1. _validate_collection_dim_result:148
Old: if not embeddings or len(embeddings) == 0:
New: if embeddings is None or len(embeddings) == 0:
The 'if not embeddings' check raises ValueError('The truth value of an
array with more than one element is ambiguous. Use a.any() or a.all()')
when 'embeddings' is a non-empty numpy array (which is the normal case
after documents are upserted). The exception is caught by the outer
'except Exception' which returns a non-ok Result, causing __init__ to
set self.collection = None. Subsequent 'get_all_indexed_paths()' then
fails with 'NoneType has no attribute get' on self.collection.get().
2. get_all_indexed_paths:334
Old: return list(set(m.get('path') for m in res['metadatas'] if m.get('path')))
New: return list(set(m['path'] for m in res['metadatas'] if m is not None and m.get('path')))
When chromadb returns 'metadatas=[None, ...]' (documents upserted
without metadata), 'm.get('path')' fails with AttributeError on the
first None element. Adds 'm is not None' guard.
Both fixes are defensive: the conditions that trigger them (orphan docs
without metadata, non-empty embeddings arrays) are normal valid
states that the old code couldn't handle.
New file: tests/test_rag_sync_none_error.py
3 unit tests covering both bugs:
- test_dim_check_does_not_raise_on_non_empty_ndarray
- test_get_all_indexed_paths_handles_none_metadata
- test_get_all_indexed_paths_returns_paths_with_metadata
Verified:
- 3/3 focused tests pass
- test_rag_phase4_final_verify.py::test_phase4_final_verify PASSES (was failing)
- test_rag_phase4_stress.py::test_rag_large_codebase_verification_sim PASSES (was failing)
- test_rag_visual_sim.py::test_rag_full_lifecycle_sim PASSES (still passing)
The wipe path called self._init_vector_store() which re-invoked
_validate_collection_dim, causing infinite recursion (RecursionError)
when the dim mismatch test ran with the mock embedding provider.
Re-initialize the vector store INLINE after the rmtree wipe so the
fresh collection is created without going through the validator
again.
When the existing collection has embeddings from a different
embedding provider (e.g. Gemini 3072-dim vs local 384-dim), the
prior approach of calling client.delete_collection() fails with
'RustBindingsAPI object has no attribute bindings' in chromadb 1.5.x
when the underlying state is corrupted. rmtree is reliable and
re-creates a fresh empty collection.
Also fixes:
- 'The truth value of an empty array is ambiguous' on numpy 2.x
by using try/except around len() instead of truthiness check
- WinError 32 on rmtree by closing the chroma client first
Verified: tests/test_rag_phase4_final_verify.py passes in isolation
in 7.75s after this fix. The test still fails in batch context due
to a separate io_pool race condition (multiple _sync_rag_engine
calls collide when the test sets rag_enabled, rag_source, and
rag_emb_provider in sequence). The race is in app_controller.py
and is out of scope for this defensive fix.
Note: tests/test_rag_engine.py has explicit unit tests for
test_rag_collection_dim_mismatch_recreates_collection and
test_rag_collection_dim_match_preserves_collection which
exercise this code path.
RAGEngine.index_file silently returns when the joined base_dir+file_path
doesn't exist. This caused the RAG batch test to fail with 0 indexed
documents when the live_gui subprocess's active_project_root resolved
to a parent dir (e.g. tests/artifacts/) instead of the workspace
(tests/artifacts/live_gui_workspace/).
The fix: if the primary path doesn't exist, try CWD+file_path. The
base_dir takes priority; CWD is a safety net for relative-path
resolution across the spawn CWD boundary.
This is a defensive fix at the rag_engine layer. It does NOT fix the
underlying path-leakage issue in tests/conftest.py (hardcoded
Path('tests/artifacts/live_gui_workspace')) which needs a proper
fixture refactor. The RAG test still fails in batch due to that
deeper issue, documented in docs/reports/rag_test_batch_failure_status_20260609_pm3.md.
Behavior:
- base_dir+file_path exists: indexed from base_dir (unchanged)
- base_dir+file_path missing, CWD+file_path exists: indexed from CWD (new)
- Both missing: silently returns (unchanged)
Verified: tests/test_rag_index_file_path_fallback.py (3 tests, all pass)
- test_index_file_finds_file_via_cwd_fallback
- test_index_file_uses_base_dir_first
- test_index_file_silently_returns_when_no_match
Note: test file was removed before commit because it was being
abandoned along with the broader path-hygiene refactor. The fix
itself is preserved in src/rag_engine.py.
- Improved AppController.ai_status to prevent overwriting 'sending...' with 'models loaded'.
- Enhanced est_rag_phase4_stress.py with robust polling and increased timeout.
- Synchronized App and AppController history objects to ensure consistent view.
- Fixed
ullcontext NameError in gui_2.py.
- Corrected TestMMAApprovalIndicators to call real rendering methods on mock app.
- Updated est_history_manager.py to provide required context_files argument to UISnapshot.
- Stabilized est_z_negative_flows.py with robust polling for terminal response status and corrected field names.
- Cleaned up debug logging in
ag_engine.py and pp_controller.py.
- Fixed circular import in chromadb by using lazy imports in
ag_engine.py.
- Moved RAG engine initialization to background threads in AppController to avoid blocking UI.
- Added _rag_engine_lock to prevent race conditions during engine re-initialization.
- Updated Gemini embedding model to gemini-embedding-001 (available) from ext-embedding-004 (not found).
- Fixed _rebuild_rag_index to use fresh
ag_engine instance from self in every iteration.
- Optimized est_rag_phase4_final_verify.py and est_rag_phase4_stress.py to wait for RAG sync before continuing.
- Added dummy embedding fallback in LocalEmbeddingProvider if sentence-transformers fails to load.
ed
r00tz