Private
Public Access
0
0

docs(workflow): add Skip-Marker Policy section

Per 2026-06-07 user feedback during test_suite cleanup:
"if the intent is to annotate a known failure, fine. But that
known failure must be addressed with priority."

New section between "Per-Task Decision Protocol" and
"Documentation Refresh Protocol" makes the policy explicit:

- Skip markers are DOCUMENTATION, not avoidance
- They're useful for opt-in integration tests, unimplemented
  features, or feature-flag-gated code
- They're NOT useful for pre-existing failures, "I don't
  understand this" issues, or racy tests the agent doesn't want
  to debug
- When adding a marker, MUST document the underlying issue AND
  what the fix would be
- When the fix is in-session reachable, FIX IT INSTEAD of
  skipping — limited context is not an excuse

Includes a 4-question review checklist before adding a skip.
References the existing AGENTS.md "Use skip markers as excuse to
AVOID" rule so the two policies don't drift.
This commit is contained in:
2026-06-07 16:57:54 -04:00
parent ff523f7e6e
commit c9c5535889
+31
View File
@@ -551,6 +551,37 @@ When the implementing agent encounters a decision not covered by the plan:
---
## Skip-Marker Policy: Documentation, Not Avoidance
`@pytest.mark.skip(reason=...)` is **documentation of a known failure**, not a way to avoid fixing the underlying bug. Skip markers are useful for:
- **Opt-in integration tests** that require external resources (a real API key, a live provider, a specific env var). Use `@pytest.mark.skipif(...)` with an env-var gate so the test runs when the resource is available and skips by default.
- **Tests for features that don't exist yet** (planned but not implemented).
- **Tests for features behind a feature flag** that's currently off.
Skip markers are NOT useful for:
- **Pre-existing failing tests** (a test that "used to pass" or "was supposed to pass but the underlying code regressed"). The underlying code/test should be fixed in-session.
- **Tests that the agent doesn't understand** ("I don't know how to fix this, so I'll skip it"). Escalate to a Tier 4 QA agent for analysis, or ask the user.
- **Tests with racy assertions that the agent doesn't want to debug** (e.g., a `time.sleep(0.5)` would fix it). Fix the race, don't skip.
**When you add a skip marker, you MUST also:**
1. Document the underlying issue in the `reason=` string (one or two sentences).
2. State what the fix would be (file:line or a one-line description).
3. Commit the skip with a follow-up note in the commit body that records the underlying issue, so the next agent (or future self after compaction) can find it via `git log --oneline --grep "skip"`.
**When the underlying issue is fixable in-session, FIX IT INSTEAD of adding a skip marker.** Limited context is not an excuse: the agent may not know whether the fix is "important" or "easy" until it tries. A skip marker that never gets revisited is a silent test-suite rot.
**Review checklist before adding a skip marker:**
- [ ] Is this a known-bad infrastructure issue (env-var gated)? Use `@pytest.mark.skipif` instead.
- [ ] Is this a feature not yet implemented? If so, the feature should be a TODO, not a skip.
- [ ] Can the test be fixed in < 30 minutes of investigation? If yes, fix it.
- [ ] If the fix is too large, is the underlying issue tracked elsewhere (a conductor track, a TODO in the code)?
Reference: AGENTS.md "Critical Anti-Patterns" section "Use skip markers as excuse to AVOID" (added 2026-06-07).
---
## Documentation Refresh Protocol
Architectural refactor tracks often change the *shape* of modules the existing docs describe. After a track ships, the affected guides may be partly out of date.