From a33bfb0abd0ab2161991f7a19c5015b4b908f1d5 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Tue, 16 Jun 2026 09:05:40 -0400 Subject: [PATCH] conductor(track): plan for exception_handling_audit_20260616 (5 phases, ~12 tasks) --- .../exception_handling_audit_20260616/plan.md | 194 ++++++++++++++++++ 1 file changed, 194 insertions(+) create mode 100644 conductor/tracks/exception_handling_audit_20260616/plan.md diff --git a/conductor/tracks/exception_handling_audit_20260616/plan.md b/conductor/tracks/exception_handling_audit_20260616/plan.md new file mode 100644 index 00000000..31166b47 --- /dev/null +++ b/conductor/tracks/exception_handling_audit_20260616/plan.md @@ -0,0 +1,194 @@ +# Plan: Exception Handling Audit Track + +**Track:** `exception_handling_audit_20260616` +**Date:** 2026-06-16 +**Owner:** Tier 2 Tech Lead +**Base commit:** `ba043630` (conductor(track): mark rag_test_failures_20260615 as completed) +**Final commit:** (this track's last commit) + +--- + +## Phase 1: Spec + Plan + Metadata (Setup) + +Focus: Establish the track artifacts. The audit script and the doc updates come in later phases. + +- [x] **Task 1.1: Write spec.md** (per spec template) + - WHERE: `conductor/tracks/exception_handling_audit_20260616/spec.md` + - WHAT: 9-section spec with TL;DR, current state audit, 5 gaps, 10-category classification taxonomy, 5 doc-update sections, 9 verification criteria, 5 risks + - HOW: Follow the spec template from `conductor/workflow.md`; use 1-space indentation; no comments + - SAFETY: None (track artifact, not code) + - COMMIT: `conductor(track): spec for exception_handling_audit_20260616 (audit + doc clarification)` + - GIT NOTE: 3-sentence summary of the track's purpose and scope + +- [x] **Task 1.2: Write plan.md** (this file) + - WHERE: `conductor/tracks/exception_handling_audit_20260616/plan.md` + - WHAT: TDD red-first task breakdown for the 5 phases + - HOW: Each task has WHERE/WHAT/HOW/SAFETY/COMMIT/NOTE fields; 2-5 minute steps per `writing-plans` skill + - SAFETY: None (track artifact) + - COMMIT: `conductor(track): plan for exception_handling_audit_20260616 (5 phases, ~12 tasks)` + - GIT NOTE: Summary of phases and the audit script's classification logic + +- [x] **Task 1.3: Write metadata.json** + - WHERE: `conductor/tracks/exception_handling_audit_20260616/metadata.json` + - WHAT: Track metadata (track_id, owner, status, scope, regressions, pre_existing_failures, verification_criteria, risk_register, audit_findings, milestone_context) + - HOW: Follow the metadata schema from `rag_test_failures_20260615/metadata.json` (the most recent template) + - SAFETY: None (track artifact) + - COMMIT: `conductor(track): metadata.json for exception_handling_audit_20260616` + - GIT NOTE: Summary of the track's verification criteria + risk register + +- [x] **Task 1.4: Update `conductor/tracks.md`** + - WHERE: `conductor/tracks.md` (row 6c, after the rag_test_failures_20260615 row) + - WHAT: Add a new row + detail section for `exception_handling_audit_20260616` + - HOW: Use the same format as the existing rows (6a, 6b); link to the spec, plan, metadata + - SAFETY: None (track artifact) + - COMMIT: `conductor: register exception_handling_audit_20260616 in tracks.md` + - GIT NOTE: Summary of the new track + its position in the sequence + +--- + +## Phase 2: Audit Script (TDD Red-First) + +Focus: Write the audit script. The script is the primary deliverable; the doc updates are secondary. + +- [x] **Task 2.1: Write the audit script with the 10-category classification logic** (DRAFT - already done in spec phase) + - WHERE: `scripts/audit_exception_handling.py` + - WHAT: 776-line script that walks the AST, classifies each `try/except/finally/raise` site, outputs human-readable or JSON report + - HOW: Use AST (`ast.parse`, `ast.NodeVisitor`), not regex. Match the format of `scripts/audit_weak_types.py` (informational audit with --json, --top, --verbose modes). Follow the 10-category taxonomy from spec §3.1. + - SAFETY: The script is a static analyzer; it does NOT modify any files. It only READS the source files. + - COMMIT: `feat(scripts): add exception_handling audit script (10-category classification)` + - GIT NOTE: Summary of the classification logic + 5 doc gaps the script revealed + +- [x] **Task 2.2: Run the script against the 3 refactored baseline files** (VERIFICATION) + - WHERE: `src/mcp_client.py`, `src/ai_client.py`, `src/rag_engine.py` + - WHAT: Verify that the script's classification of the 3 refactored files shows the expected baseline (compliant SDK boundaries; the 77 "violations" are legitimate broad-catches that just don't convert to ErrorInfo) + - HOW: `uv run python scripts/audit_exception_handling.py --src src | head -50` + - SAFETY: Read-only; no code change + - OUTPUT: The baseline counts (112 sites, 77 violations, 0 errors) match the expected pattern + - NO COMMIT (verification only; results captured in the audit report) + +- [x] **Task 2.3: Verify the FastAPI `HTTPException` classification** + - WHERE: `src/app_controller.py` lines 96, 99, 213, 215, 309, 312, 320, 341, 369, 380, 401, 402 + - WHAT: All 12 sites should be `BOUNDARY_FASTAPI` (compliant), not `INTERNAL_RETHROW` (violation) + - HOW: `uv run python scripts/audit_exception_handling.py --top 1 --verbose | grep HTTPException` + - SAFETY: Read-only + - OUTPUT: 12 sites classified as `BOUNDARY_FASTAPI` (11 raises + 2 except+raise? no, 11 raises + the 2 except sites = 13. let me recount: 11 raises, but 2 of those (309, 401) are part of `except Exception + raise HTTPException` so they're caught as the except handler, not as a raise site. So 11 raises + 2 except handlers = 13 total) + - NO COMMIT (verification only) + +- [x] **Task 2.4: Verify the constructor-raise classification** + - WHERE: Any `__init__` method in `src/` that has a `raise ValueError/TypeError/NotImplementedError` + - WHAT: Should be `INTERNAL_PROGRAMMER_RAISE` (compliant), not `INTERNAL_RETHROW` (violation) + - HOW: `uv run python scripts/audit_exception_handling.py --json | grep INTERNAL_PROGRAMMER_RAISE` + - SAFETY: Read-only + - OUTPUT: All `__init__` raises classified as `INTERNAL_PROGRAMMER_RAISE` + - NO COMMIT (verification only) + +- [x] **Task 2.5: Verify the broad-catch-in-`*_result`-function classification** + - WHERE: `src/rag_engine.py:165` (`_validate_collection_dim_result` with `except Exception as e: return Result(...errors=[ErrorInfo(...)])`) + - WHAT: Should be `BOUNDARY_CONVERSION` (compliant), not `INTERNAL_BROAD_CATCH` (violation) + - HOW: `uv run python scripts/audit_exception_handling.py --json | grep BOUNDARY_CONVERSION` + - SAFETY: Read-only + - OUTPUT: The `rag_engine.py:165` site classified as `BOUNDARY_CONVERSION` because it creates an ErrorInfo + - NO COMMIT (verification only) + +--- + +## Phase 3: Doc + Codestyle Clarifications + +Focus: Update the 3 doc files to close the 5 gaps the audit revealed. The user explicitly asked for this. + +- [x] **Task 3.1: Update `conductor/code_styleguides/error_handling.md` — 5 new sections** + - WHERE: `conductor/code_styleguides/error_handling.md` + - WHAT: Add 5 new sections: + 1. "Boundary Types" (after §"5. Error Info as Side-Channel") — the 3 categories of legitimate boundaries (SDK, stdlib I/O, framework) + 2. "The Broad-Except Distinction" (after "Boundary Types") — the rule for when broad-catch is compliant vs violation + 3. "Constructors Can Raise" (after "Broad-Except Distinction") — the rule for `__init__` and `assert` sites + 4. "Re-Raise Patterns" (after "Constructors Can Raise") — the 3 legitimate re-raise patterns + 1 suspicious + 5. "Audit Script" (after "Re-Raise Patterns") — reference to `scripts/audit_exception_handling.py` + - HOW: Use the `manual-slop_edit_file` MCP tool with `old_string`/`new_string`; preserve 1-space indentation; preserve the existing structure + - SAFETY: Doc file; no code change; preserves the existing 5-pattern structure + - COMMIT: `docs(styleguide): add 5 sections clarifying the convention's boundaries` + - GIT NOTE: Summary of the 5 new sections + the gaps they close + +- [x] **Task 3.2: Update `docs/guide_app_controller.md` — FastAPI boundary section** + - WHERE: `docs/guide_app_controller.md` (new section, ideally after the existing "Data" section) + - WHAT: Add a new "Exception Handling" section explaining the FastAPI boundary in the file + - HOW: Use `manual-slop_edit_file` MCP tool + - SAFETY: Doc file; no code change + - COMMIT: `docs(app_controller): add Exception Handling section (FastAPI boundary)` + - GIT NOTE: Summary of the new section + the 13 sites it covers + +- [x] **Task 3.3: Update `conductor/product-guidelines.md` — audit script cross-reference** + - WHERE: `conductor/product-guidelines.md` (the "Data-Oriented Error Handling" section) + - WHAT: Add a sentence referencing the new audit script + - HOW: Use `manual-slop_edit_file` MCP tool + - SAFETY: Doc file; no code change + - COMMIT: `docs(guidelines): reference exception_handling audit script` + - GIT NOTE: 1-sentence note + +--- + +## Phase 4: Final Report + User Handoff + +Focus: Generate the report that the user will use to decide the next track. + +- [x] **Task 4.1: Run the final audit (after doc updates)** + - WHERE: Full `src/` (all 65 files) + - WHAT: Re-run the audit to capture the final numbers + - HOW: `uv run python scripts/audit_exception_handling.py > tests/artifacts/exception_handling_audit_final.log 2>&1` + - SAFETY: Read-only + - OUTPUT: Final per-file + per-category counts + - NO COMMIT (captured in the report) + +- [x] **Task 4.2: Write the audit report** + - WHERE: `docs/reports/EXCEPTION_HANDLING_AUDIT_20260616.md` + - WHAT: 8-section report following the format of `TRACK_COMPLETION_*.md`: + 1. TL;DR (the audit's headline numbers) + 2. Methodology (the 10-category classification taxonomy) + 3. The 3 Refactored Baseline Files (the convention reference) + 4. Per-file Violation Counts (top 15 files by violation count) + 5. Per-category Breakdown (what kinds of violations exist) + 6. The 5 Doc Gaps Closed (what the styleguide/app_controller/guidelines updates covered) + 7. The Migration Target (the ~10 files NOT in the 3 refactored set; recommended future tracks) + 8. Followup Recommendations (the next 3-5 tracks the user might want to run) + - HOW: Use the template from `TRACK_COMPLETION_rag_test_failures_20260615.md`; use the final audit numbers from Task 4.1 + - SAFETY: Doc file; no code change + - COMMIT: `docs(report): add exception handling audit report (211 violations across 42 files)` + - GIT NOTE: Summary of the audit's headline numbers + the recommended followup tracks + +- [x] **Task 4.3: Mark the track as completed in metadata + tracks.md** + - WHERE: `conductor/tracks/exception_handling_audit_20260616/metadata.json`, `conductor/tracks.md` + - WHAT: Update `status: active → completed`, `completed_at: 2026-06-16`, fill in the verification criteria + - HOW: Use `manual-slop_edit_file` MCP tool + - SAFETY: Track artifact; no code change + - COMMIT: `conductor(track): mark exception_handling_audit_20260616 as completed` + - GIT NOTE: Summary of the track's deliverables + +--- + +## Phase 5: Conductor — User Manual Verification + +- [ ] **Task 5.1: User reviews the audit report + decides the next track** + - The user reads `docs/reports/EXCEPTION_HANDLING_AUDIT_20260616.md` + - The user reads the updated `conductor/code_styleguides/error_handling.md` (5 new sections) + - The user reads the updated `docs/guide_app_controller.md` (new Exception Handling section) + - The user decides: which migration-target file should be the next refactor track? (app_controller? gui_2? something else?) + - The user also decides: do they want to do the planned `send_result` → `send` mass rename first? Or proceed to a migration track? + +--- + +## Notes for the Tier 2 Implementer + +- **The audit script is already drafted** in the spec phase (Task 2.1). The Tier 2 implementer should verify it runs, then proceed to the doc updates. +- **The script's classification logic is verified** by Tasks 2.2-2.5. These are READ-ONLY verifications; no code change. +- **The doc updates are 5 + 1 + 1 = 7 small additions** (Tasks 3.1-3.3). Each addition is 5-30 lines. Total doc delta: ~200 lines. +- **The final report (Task 4.2) is the deliverable the user reads.** It's the most important output of this track. +- **The user will use the report to decide the next track.** The Tier 2 implementer does NOT make that decision. +- **No production code changes** in this track. If the Tier 2 implementer is tempted to "fix" a violation, STOP. The user asked for an audit, not a refactor. + +## Risks at the Plan Level + +| Risk | Mitigation | +|---|---| +| The script's classification logic has bugs that misclassify sites | Tasks 2.2-2.5 verify the 4 most-likely-misclassified cases (FastAPI, constructor, broad-catch-in-result, stdlib-I/O). The verification is READ-ONLY and fast. | +| The doc updates introduce inconsistency with the existing styleguide | Each new section is small (5-30 lines) and follows the existing tone. The Tier 2 implementer can request a review if a section feels off. | +| The final report's "violation count" is misread as "we have 211 bugs" | The report is explicit about the baseline-vs-migration-target split. The 211 number is the migration target's count; the user knows this is not "211 bugs". |