diff --git a/conductor/tracks/superpowers_review_20260619/report.md b/conductor/tracks/superpowers_review_20260619/report.md index 3e41ad3a..f452e4cc 100644 --- a/conductor/tracks/superpowers_review_20260619/report.md +++ b/conductor/tracks/superpowers_review_20260619/report.md @@ -311,7 +311,77 @@ The project follows the discipline with significant elaborations (Phase Completi ## 6. Systematic Debugging - +## 6. Systematic Debugging + +### 6.1 What the skill prescribes + +The `systematic-debugging` skill is the canonical anti-pattern against "random fixes waste time and create new bugs." The Iron Law: "NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST." The skill's 4-Phase model is exhaustive: + +**Phase 1 — Root Cause Investigation:** (1) Read error messages carefully; (2) Reproduce consistently; (3) Check recent changes (git diff, recent commits); (4) Gather evidence in multi-component systems (log what enters/exits each boundary); (5) Trace data flow (root-cause-tracing.md for the backward-tracing technique). + +**Phase 2 — Pattern Analysis:** (1) Find working examples; (2) Compare against references (read reference implementation COMPLETELY); (3) Identify differences; (4) Understand dependencies. + +**Phase 3 — Hypothesis and Testing:** (1) Form single hypothesis ("I think X is the root cause because Y"); (2) Test minimally (smallest possible change, one variable); (3) Verify before continuing; (4) When you don't know, say so. + +**Phase 4 — Implementation:** (1) Create failing test case (TDD); (2) Implement single fix (no bundled refactoring); (3) Verify fix; (4) If fix doesn't work: STOP and count — if <3, return to Phase 1; if ≥3, STOP and question architecture (Phase 4.5). + +The Red Flags are extensive: "Quick fix for now, investigate later"; "Just try changing X and see if it works"; "Add multiple changes, run tests"; "Skip the test, I'll manually verify"; "It's probably X, let me fix that"; "I don't fully understand but this might work"; "Pattern says X but I'll adapt it differently"; "One more fix attempt" (when already tried 2+); each fix reveals new problem in different place. + +### 6.2 Mapping to the project's existing pattern + +The project has a comprehensive Process Anti-Patterns section in `AGENTS.md` and a "Process Anti-Patterns (Added 2026-06-09)" section in `conductor/workflow.md`. Together they map directly to the skill's 4-Phase model: + +| Skill rule | Project equivalent | Where | +|---|---|---| +| Phase 1: "Read Error Messages Carefully" | The "Deduction Loop (kill it)" anti-pattern: "You are allowed to run a failing test at most 2 times in a single investigation. After the 2nd failure, STOP running the test. Read the relevant source code (get_file_slice or py_get_skeleton), predict the failure mode from the code, and instrument ALL the relevant state in one pass before the next run." | `AGENTS.md` "Process Anti-Patterns" §1 | +| Phase 1: "Reproduce Consistently" | The "Isolated-Pass Verification Fallacy" rule (per Section 5 above): tests passing in isolation but failing in batch is FAILING. | `conductor/workflow.md` §"Isolated-Pass Verification Fallacy" | +| Phase 1: "Check Recent Changes" | The "Inherited-Cruft Pattern (kill it)" anti-pattern: "If the file is already in a broken state from a previous session, the FIRST thing you do is ask the user: 'this file is in a broken state from a previous agent. do you want me to (a) revert the working tree and start from a clean baseline, (b) finish the previous agent's intent, or (c) abandon the work entirely?'" | `AGENTS.md` "Process Anti-Patterns" §4 | +| Phase 1: "Gather Evidence in Multi-Component Systems" | The Deduction Loop rule's "instrument ALL the relevant state in one pass before the next run" + `scripts/audit_*` scripts (audit_main_thread_imports, audit_weak_types, etc.) | `AGENTS.md` §1 + `scripts/audit_*` | +| Phase 2: "Find Working Examples" | The project's MCP tools (e.g., `py_get_skeleton`, `py_get_code_outline`) enable finding similar patterns in the codebase; the project's "Current State Audit" step in tier-1 specs | `conductor/workflow.md` §"The Surgical Methodology" | +| Phase 3: "Form Single Hypothesis" | The Deduction Loop's "predict the failure mode from the code" step enforces single-hypothesis discipline | `AGENTS.md` §1 | +| Phase 4: "Create Failing Test Case" | TDD (per Section 4 above); the project uses tier-3 worker dispatches to write the failing test | `conductor/workflow.md` §"Task Workflow" step 4 | +| Phase 4: "If 3+ Fixes Failed: Question Architecture" | The Deduction Loop's "If you've tried 3 times and the test still fails, STOP and report to the user" + the "Inherited-Cruft Pattern" + the Surrender pattern's "5 things to do before surrendering" | `AGENTS.md` §1 + §4 + §6 | +| "Just try this first, then investigate" | The Deduction Loop's "You are allowed to run a failing test at most 2 times" — strictly bound the trial-and-error budget | `AGENTS.md` §1 | +| "Multiple fixes at once saves time" | The Deduction Loop's "instrument ALL the relevant state in one pass before the next run" — one-pass instrumentation, then run ONCE more | `AGENTS.md` §1 | +| "I see the problem, let me fix it" | The "Report-Instead-of-Fix Pattern (kill it)": "A status report is allowed only when: You have actually tried the fix and it failed with evidence, OR You are blocked on a decision the user must make. A status report is NOT allowed when: You are avoiding a hard problem by writing prose about it." | `AGENTS.md` §2 | +| "I'm confident" | The Deduction Loop's "predict the failure mode from the code" — confidence is replaced with code-derived prediction | `AGENTS.md` §1 | +| "I'm tired" / "Manager wants it fixed NOW" | n/a — the project doesn't have this explicit anti-pattern; the user catches it via the report-instead-of-fix rule | — | + +### 6.3 Where the project already follows the discipline + +- **The Deduction Loop rule** is the project's most explicit anti-rationalization rule. It says "run at most 2 times, then STOP, then predict from code, then instrument once, then run once more." This maps directly to the skill's Phase 1 "Reproduce Consistently" + Phase 3 "Test Minimally". +- **The Report-Instead-of-Fix Pattern (kill it)** maps to the skill's "Don't pretend to know" rule. The project's rule is stricter: a 200-line status report is forbidden; a 5-10-sentence status report is allowed only when blocked. +- **The Inherited-Cruft Pattern (kill it)** maps to the skill's "Check Recent Changes" Phase 1 rule. The project adds a layer: if the file is broken from a previous session, ASK the user before fixing. +- **The Verbose-Commit-Message Pattern (kill it)** is project-specific: commit messages are 1-3 sentences, not 50-line reports. This is the "stop writing prose about work" anti-pattern. +- **The Isolated-Pass Verification Fallacy** is the project's anti-pattern for Phase 4 step 3 ("Verify Fix"). It says the fix isn't verified until it passes in batch. + +### 6.4 Where the project doesn't follow the discipline + +- **"If 3+ Fixes Failed: Question Architecture"** is partially followed. The Deduction Loop says "STOP and report to the user" after 3 fixes, but doesn't explicitly say "question architecture". The project does have the "Inherited-Cruft Pattern" rule (which is a form of architecture questioning) but it's framed around broken-file state, not failed-fix state. +- **"Quick fix for now, investigate later"** is not explicitly addressed. The Deduction Loop catches the symptom (running the test 3 times) but not the rationalization ("I'll investigate later"). +- **"I don't fully understand but this might work"** is caught by the Report-Instead-of-Fix rule (status reports are forbidden when avoiding a hard problem), but there's no explicit "say you don't understand" rule. +- **The skill's "Defense in Depth" + "Condition-Based Waiting" techniques** are mentioned in the project (the live_gui test fixture uses condition polling; the result_types.py uses defense-in-depth Result[T]) but they're not codified as project anti-patterns. +- **The skill's "Trace Data Flow" (root-cause-tracing.md)** is implicit in the project's MCP tool usage (`py_find_usages`, `py_get_definition`, `py_get_imports`) but no explicit "use the tracing technique" rule. + +### 6.5 Recommendations summary + +The project follows the systematic-debugging discipline with significant elaborations (Deduction Loop, Report-Instead-of-Fix, Inherited-Cruft, Verbose-Commit-Message, Isolated-Pass Fallacy). The project goes BEYOND the skill in catching rationalization patterns. The deferred rebuild may want to: + +- **MEDIUM:** Add explicit "Question Architecture" rule to AGENTS.md §"Process Anti-Patterns": "After 3 failed fix attempts, stop and question the architecture before attempting Fix #4. The pattern may be fundamentally wrong; discuss with the user before continuing." +- **LOW:** Add an explicit "Defense in Depth" + "Condition-Based Waiting" reference section to AGENTS.md or to a new `conductor/code_styleguides/debugging.md` styleguide. + +**Verdict.** + +| Field | Value | +|---|---| +| **Primary** | `PARITY` | +| **Integration tag** | `INTEGRATED` | +| **Section size** | deep-dive | +| **Cross-refs** | nagent_review_20260608 §10 ("durable debugging" — the project's test discipline); fable_review_20260617 §13 (project's debugging is "genuinely useful" pattern); intent_dsl_survey_20260612 §6 (debugging is one of the project's "AI-agent properties") | + +**Rationale.** The project's AGENTS.md + workflow.md Process Anti-Patterns section is a faithful re-encoding of the skill's 4-Phase model with five project-specific elaborations. The Deduction Loop + Report-Instead-of-Fix + Inherited-Cruft rules map directly to the skill's Phase 1, Phase 3, and Phase 4. The skill's "Question Architecture" rule is partially followed but could be explicit. + +**Recommended change.** Add explicit "Question Architecture" rule to AGENTS.md §"Process Anti-Patterns" (mirror the skill's Phase 4.5); consider a `conductor/code_styleguides/debugging.md` for the supporting techniques (defense-in-depth, condition-based waiting, root-cause-tracing). ## 7. Subagent-Driven Development