Private
Public Access
conductor(track): plan for exception_handling_audit_20260616 (5 phases, ~12 tasks)
This commit is contained in:
@@ -0,0 +1,194 @@
|
||||
# Plan: Exception Handling Audit Track
|
||||
|
||||
**Track:** `exception_handling_audit_20260616`
|
||||
**Date:** 2026-06-16
|
||||
**Owner:** Tier 2 Tech Lead
|
||||
**Base commit:** `ba043630` (conductor(track): mark rag_test_failures_20260615 as completed)
|
||||
**Final commit:** (this track's last commit)
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Spec + Plan + Metadata (Setup)
|
||||
|
||||
Focus: Establish the track artifacts. The audit script and the doc updates come in later phases.
|
||||
|
||||
- [x] **Task 1.1: Write spec.md** (per spec template)
|
||||
- WHERE: `conductor/tracks/exception_handling_audit_20260616/spec.md`
|
||||
- WHAT: 9-section spec with TL;DR, current state audit, 5 gaps, 10-category classification taxonomy, 5 doc-update sections, 9 verification criteria, 5 risks
|
||||
- HOW: Follow the spec template from `conductor/workflow.md`; use 1-space indentation; no comments
|
||||
- SAFETY: None (track artifact, not code)
|
||||
- COMMIT: `conductor(track): spec for exception_handling_audit_20260616 (audit + doc clarification)`
|
||||
- GIT NOTE: 3-sentence summary of the track's purpose and scope
|
||||
|
||||
- [x] **Task 1.2: Write plan.md** (this file)
|
||||
- WHERE: `conductor/tracks/exception_handling_audit_20260616/plan.md`
|
||||
- WHAT: TDD red-first task breakdown for the 5 phases
|
||||
- HOW: Each task has WHERE/WHAT/HOW/SAFETY/COMMIT/NOTE fields; 2-5 minute steps per `writing-plans` skill
|
||||
- SAFETY: None (track artifact)
|
||||
- COMMIT: `conductor(track): plan for exception_handling_audit_20260616 (5 phases, ~12 tasks)`
|
||||
- GIT NOTE: Summary of phases and the audit script's classification logic
|
||||
|
||||
- [x] **Task 1.3: Write metadata.json**
|
||||
- WHERE: `conductor/tracks/exception_handling_audit_20260616/metadata.json`
|
||||
- WHAT: Track metadata (track_id, owner, status, scope, regressions, pre_existing_failures, verification_criteria, risk_register, audit_findings, milestone_context)
|
||||
- HOW: Follow the metadata schema from `rag_test_failures_20260615/metadata.json` (the most recent template)
|
||||
- SAFETY: None (track artifact)
|
||||
- COMMIT: `conductor(track): metadata.json for exception_handling_audit_20260616`
|
||||
- GIT NOTE: Summary of the track's verification criteria + risk register
|
||||
|
||||
- [x] **Task 1.4: Update `conductor/tracks.md`**
|
||||
- WHERE: `conductor/tracks.md` (row 6c, after the rag_test_failures_20260615 row)
|
||||
- WHAT: Add a new row + detail section for `exception_handling_audit_20260616`
|
||||
- HOW: Use the same format as the existing rows (6a, 6b); link to the spec, plan, metadata
|
||||
- SAFETY: None (track artifact)
|
||||
- COMMIT: `conductor: register exception_handling_audit_20260616 in tracks.md`
|
||||
- GIT NOTE: Summary of the new track + its position in the sequence
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Audit Script (TDD Red-First)
|
||||
|
||||
Focus: Write the audit script. The script is the primary deliverable; the doc updates are secondary.
|
||||
|
||||
- [x] **Task 2.1: Write the audit script with the 10-category classification logic** (DRAFT - already done in spec phase)
|
||||
- WHERE: `scripts/audit_exception_handling.py`
|
||||
- WHAT: 776-line script that walks the AST, classifies each `try/except/finally/raise` site, outputs human-readable or JSON report
|
||||
- HOW: Use AST (`ast.parse`, `ast.NodeVisitor`), not regex. Match the format of `scripts/audit_weak_types.py` (informational audit with --json, --top, --verbose modes). Follow the 10-category taxonomy from spec §3.1.
|
||||
- SAFETY: The script is a static analyzer; it does NOT modify any files. It only READS the source files.
|
||||
- COMMIT: `feat(scripts): add exception_handling audit script (10-category classification)`
|
||||
- GIT NOTE: Summary of the classification logic + 5 doc gaps the script revealed
|
||||
|
||||
- [x] **Task 2.2: Run the script against the 3 refactored baseline files** (VERIFICATION)
|
||||
- WHERE: `src/mcp_client.py`, `src/ai_client.py`, `src/rag_engine.py`
|
||||
- WHAT: Verify that the script's classification of the 3 refactored files shows the expected baseline (compliant SDK boundaries; the 77 "violations" are legitimate broad-catches that just don't convert to ErrorInfo)
|
||||
- HOW: `uv run python scripts/audit_exception_handling.py --src src | head -50`
|
||||
- SAFETY: Read-only; no code change
|
||||
- OUTPUT: The baseline counts (112 sites, 77 violations, 0 errors) match the expected pattern
|
||||
- NO COMMIT (verification only; results captured in the audit report)
|
||||
|
||||
- [x] **Task 2.3: Verify the FastAPI `HTTPException` classification**
|
||||
- WHERE: `src/app_controller.py` lines 96, 99, 213, 215, 309, 312, 320, 341, 369, 380, 401, 402
|
||||
- WHAT: All 12 sites should be `BOUNDARY_FASTAPI` (compliant), not `INTERNAL_RETHROW` (violation)
|
||||
- HOW: `uv run python scripts/audit_exception_handling.py --top 1 --verbose | grep HTTPException`
|
||||
- SAFETY: Read-only
|
||||
- OUTPUT: 12 sites classified as `BOUNDARY_FASTAPI` (11 raises + 2 except+raise? no, 11 raises + the 2 except sites = 13. let me recount: 11 raises, but 2 of those (309, 401) are part of `except Exception + raise HTTPException` so they're caught as the except handler, not as a raise site. So 11 raises + 2 except handlers = 13 total)
|
||||
- NO COMMIT (verification only)
|
||||
|
||||
- [x] **Task 2.4: Verify the constructor-raise classification**
|
||||
- WHERE: Any `__init__` method in `src/` that has a `raise ValueError/TypeError/NotImplementedError`
|
||||
- WHAT: Should be `INTERNAL_PROGRAMMER_RAISE` (compliant), not `INTERNAL_RETHROW` (violation)
|
||||
- HOW: `uv run python scripts/audit_exception_handling.py --json | grep INTERNAL_PROGRAMMER_RAISE`
|
||||
- SAFETY: Read-only
|
||||
- OUTPUT: All `__init__` raises classified as `INTERNAL_PROGRAMMER_RAISE`
|
||||
- NO COMMIT (verification only)
|
||||
|
||||
- [x] **Task 2.5: Verify the broad-catch-in-`*_result`-function classification**
|
||||
- WHERE: `src/rag_engine.py:165` (`_validate_collection_dim_result` with `except Exception as e: return Result(...errors=[ErrorInfo(...)])`)
|
||||
- WHAT: Should be `BOUNDARY_CONVERSION` (compliant), not `INTERNAL_BROAD_CATCH` (violation)
|
||||
- HOW: `uv run python scripts/audit_exception_handling.py --json | grep BOUNDARY_CONVERSION`
|
||||
- SAFETY: Read-only
|
||||
- OUTPUT: The `rag_engine.py:165` site classified as `BOUNDARY_CONVERSION` because it creates an ErrorInfo
|
||||
- NO COMMIT (verification only)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Doc + Codestyle Clarifications
|
||||
|
||||
Focus: Update the 3 doc files to close the 5 gaps the audit revealed. The user explicitly asked for this.
|
||||
|
||||
- [x] **Task 3.1: Update `conductor/code_styleguides/error_handling.md` — 5 new sections**
|
||||
- WHERE: `conductor/code_styleguides/error_handling.md`
|
||||
- WHAT: Add 5 new sections:
|
||||
1. "Boundary Types" (after §"5. Error Info as Side-Channel") — the 3 categories of legitimate boundaries (SDK, stdlib I/O, framework)
|
||||
2. "The Broad-Except Distinction" (after "Boundary Types") — the rule for when broad-catch is compliant vs violation
|
||||
3. "Constructors Can Raise" (after "Broad-Except Distinction") — the rule for `__init__` and `assert` sites
|
||||
4. "Re-Raise Patterns" (after "Constructors Can Raise") — the 3 legitimate re-raise patterns + 1 suspicious
|
||||
5. "Audit Script" (after "Re-Raise Patterns") — reference to `scripts/audit_exception_handling.py`
|
||||
- HOW: Use the `manual-slop_edit_file` MCP tool with `old_string`/`new_string`; preserve 1-space indentation; preserve the existing structure
|
||||
- SAFETY: Doc file; no code change; preserves the existing 5-pattern structure
|
||||
- COMMIT: `docs(styleguide): add 5 sections clarifying the convention's boundaries`
|
||||
- GIT NOTE: Summary of the 5 new sections + the gaps they close
|
||||
|
||||
- [x] **Task 3.2: Update `docs/guide_app_controller.md` — FastAPI boundary section**
|
||||
- WHERE: `docs/guide_app_controller.md` (new section, ideally after the existing "Data" section)
|
||||
- WHAT: Add a new "Exception Handling" section explaining the FastAPI boundary in the file
|
||||
- HOW: Use `manual-slop_edit_file` MCP tool
|
||||
- SAFETY: Doc file; no code change
|
||||
- COMMIT: `docs(app_controller): add Exception Handling section (FastAPI boundary)`
|
||||
- GIT NOTE: Summary of the new section + the 13 sites it covers
|
||||
|
||||
- [x] **Task 3.3: Update `conductor/product-guidelines.md` — audit script cross-reference**
|
||||
- WHERE: `conductor/product-guidelines.md` (the "Data-Oriented Error Handling" section)
|
||||
- WHAT: Add a sentence referencing the new audit script
|
||||
- HOW: Use `manual-slop_edit_file` MCP tool
|
||||
- SAFETY: Doc file; no code change
|
||||
- COMMIT: `docs(guidelines): reference exception_handling audit script`
|
||||
- GIT NOTE: 1-sentence note
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Final Report + User Handoff
|
||||
|
||||
Focus: Generate the report that the user will use to decide the next track.
|
||||
|
||||
- [x] **Task 4.1: Run the final audit (after doc updates)**
|
||||
- WHERE: Full `src/` (all 65 files)
|
||||
- WHAT: Re-run the audit to capture the final numbers
|
||||
- HOW: `uv run python scripts/audit_exception_handling.py > tests/artifacts/exception_handling_audit_final.log 2>&1`
|
||||
- SAFETY: Read-only
|
||||
- OUTPUT: Final per-file + per-category counts
|
||||
- NO COMMIT (captured in the report)
|
||||
|
||||
- [x] **Task 4.2: Write the audit report**
|
||||
- WHERE: `docs/reports/EXCEPTION_HANDLING_AUDIT_20260616.md`
|
||||
- WHAT: 8-section report following the format of `TRACK_COMPLETION_*.md`:
|
||||
1. TL;DR (the audit's headline numbers)
|
||||
2. Methodology (the 10-category classification taxonomy)
|
||||
3. The 3 Refactored Baseline Files (the convention reference)
|
||||
4. Per-file Violation Counts (top 15 files by violation count)
|
||||
5. Per-category Breakdown (what kinds of violations exist)
|
||||
6. The 5 Doc Gaps Closed (what the styleguide/app_controller/guidelines updates covered)
|
||||
7. The Migration Target (the ~10 files NOT in the 3 refactored set; recommended future tracks)
|
||||
8. Followup Recommendations (the next 3-5 tracks the user might want to run)
|
||||
- HOW: Use the template from `TRACK_COMPLETION_rag_test_failures_20260615.md`; use the final audit numbers from Task 4.1
|
||||
- SAFETY: Doc file; no code change
|
||||
- COMMIT: `docs(report): add exception handling audit report (211 violations across 42 files)`
|
||||
- GIT NOTE: Summary of the audit's headline numbers + the recommended followup tracks
|
||||
|
||||
- [x] **Task 4.3: Mark the track as completed in metadata + tracks.md**
|
||||
- WHERE: `conductor/tracks/exception_handling_audit_20260616/metadata.json`, `conductor/tracks.md`
|
||||
- WHAT: Update `status: active → completed`, `completed_at: 2026-06-16`, fill in the verification criteria
|
||||
- HOW: Use `manual-slop_edit_file` MCP tool
|
||||
- SAFETY: Track artifact; no code change
|
||||
- COMMIT: `conductor(track): mark exception_handling_audit_20260616 as completed`
|
||||
- GIT NOTE: Summary of the track's deliverables
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Conductor — User Manual Verification
|
||||
|
||||
- [ ] **Task 5.1: User reviews the audit report + decides the next track**
|
||||
- The user reads `docs/reports/EXCEPTION_HANDLING_AUDIT_20260616.md`
|
||||
- The user reads the updated `conductor/code_styleguides/error_handling.md` (5 new sections)
|
||||
- The user reads the updated `docs/guide_app_controller.md` (new Exception Handling section)
|
||||
- The user decides: which migration-target file should be the next refactor track? (app_controller? gui_2? something else?)
|
||||
- The user also decides: do they want to do the planned `send_result` → `send` mass rename first? Or proceed to a migration track?
|
||||
|
||||
---
|
||||
|
||||
## Notes for the Tier 2 Implementer
|
||||
|
||||
- **The audit script is already drafted** in the spec phase (Task 2.1). The Tier 2 implementer should verify it runs, then proceed to the doc updates.
|
||||
- **The script's classification logic is verified** by Tasks 2.2-2.5. These are READ-ONLY verifications; no code change.
|
||||
- **The doc updates are 5 + 1 + 1 = 7 small additions** (Tasks 3.1-3.3). Each addition is 5-30 lines. Total doc delta: ~200 lines.
|
||||
- **The final report (Task 4.2) is the deliverable the user reads.** It's the most important output of this track.
|
||||
- **The user will use the report to decide the next track.** The Tier 2 implementer does NOT make that decision.
|
||||
- **No production code changes** in this track. If the Tier 2 implementer is tempted to "fix" a violation, STOP. The user asked for an audit, not a refactor.
|
||||
|
||||
## Risks at the Plan Level
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| The script's classification logic has bugs that misclassify sites | Tasks 2.2-2.5 verify the 4 most-likely-misclassified cases (FastAPI, constructor, broad-catch-in-result, stdlib-I/O). The verification is READ-ONLY and fast. |
|
||||
| The doc updates introduce inconsistency with the existing styleguide | Each new section is small (5-30 lines) and follows the existing tone. The Tier 2 implementer can request a review if a section feels off. |
|
||||
| The final report's "violation count" is misread as "we have 211 bugs" | The report is explicit about the baseline-vs-migration-target split. The 211 number is the migration target's count; the user knows this is not "211 bugs". |
|
||||
Reference in New Issue
Block a user