ed/manual_slop
Private
Public Access
0
0
Fork 0
Code Activity

test(mma-skills): add pressure-scenario + rule-coverage tests for 5 MMA skills (25 cases)

Browse Source 
Closes recommendation #2 from superpowers_review_20260619/decisions.md (HIGH-priority).

5 test classes (one per MMA skill): TestMmaOrchestrator, TestMmaTier1Orchestrator,
TestMmaTier2TechLead, TestMmaTier3Worker, TestMmaTier4Qa.

Each test class has 5 tests (3 pressure scenarios + 2 rule-coverage assertions)
per the superpowers writing-skills skill's Discipline-Enforcing Skills
testing methodology (see C:\Users\Ed\.cache\opencode\packages\superpowers@...
\skills\writing-skills\SKILL.md §'Testing All Skill Types').

Tests are pure static-analysis of skill documents:
- Reads .agents/skills/<skill>/SKILL.md via pathlib + regex
- No live_gui dependency, no MMA execution, no sub-agent dispatches
- Run time: 3.26s for all 25 tests
- Failure mode: if a load-bearing rule is buried in prose or absent, test fails
  with a clear message naming the missing rule + why it matters

Rule coverage:
- mma-orchestrator: Surgical Spec Protocol, Pre-Delegation Checkpoint,
  Persistent Tier 2 Memory, failure_count escalation, Architecture Fallback
- mma-tier1-orchestrator: Audit-before-specifying, Spec-gaps-not-features,
  Worker-Ready Tasks, Root Cause Analysis, Reference docs
- mma-tier2-tech-lead: Atomic Per-Task Commits, TDD Enforcement,
  Persistent Context, Anti-Entropy State Audit, Surgical Delegation
- mma-tier3-worker: TDD Mandatory Enforcement, No Architectural Decisions,
  No Unrelated File Modifications, Stateless Operation, Skeleton Views
- mma-tier4-qa: Stateless Operation, No Fix Implementation, Brief Output,
  Root Cause Analysis, Diagnostic Tools

Refs:
- superpowers_review_20260619/decisions.md #2 (HIGH-priority)
- conductor/tracks/superpowers_review_apply_high_20260705/spec.md §3.2
- superpowers writing-skills skill §Testing All Skill Types
This commit is contained in:
Edward R. Gonzalez
2026-07-05 14:15:16 -04:00
parent 7d90518627
commit 670a919e4e
1 changed files with 312 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_mma_skill_discipline.py
+312
View File
@@ -0,0 +1,312 @@
"""Pressure-scenario + rule-coverage tests for the 5 MMA skills.
Per the superpowers `writing-skills` skill (Discipline-Enforcing Skills):
3+ combined-pressure scenarios per skill + 2+ rule-coverage assertions per skill.
These tests are STATIC-ANALYSIS of skill documents (text-pattern assertions).
No live_gui dependency. No MMA execution. No sub-agent dispatches.
Run time: <5 seconds.
Generated by superpowers_review_apply_high_20260705. Closes recommendation #2
from superpowers_review_20260619/decisions.md (HIGH-priority).
"""
from __future__ import annotations
import re
from pathlib import Path
import pytest
MMA_SKILLS_DIR = Path(__file__).parent.parent / ".agents" / "skills"
def _load_skill(skill_name: str) -> str:
"""Load a SKILL.md as text. Skip the test if the skill file is absent."""
path = MMA_SKILLS_DIR / skill_name / "SKILL.md"
if not path.exists():
pytest.skip(f"MMA skill not found: {skill_name} (path={path})")
return path.read_text(encoding="utf-8")
def assert_rule_present(skill_text: str, pattern: str, *, msg: str) -> None:
"""Assert a load-bearing rule pattern is PROMINENTLY documented in the skill.
Prominence = the rule uses MUST / NEVER / Required / SHOULD NOT language,
OR appears as a section header, OR is referenced via the cross-ref
mechanism in `conductor/workflow.md`.
A rule that exists only as buried prose is the actual gap the test catches.
"""
assert re.search(pattern, skill_text, re.MULTILINE | re.IGNORECASE), msg
# ============================================================
# mma-orchestrator: Tier 1+2 dispatch protocol (parent skill).
# ============================================================
class TestMmaOrchestrator:
"""mma-orchestrator: Tier 1+2 dispatch protocol (parent skill)."""
def test_surgical_spec_protocol_required(self):
"""Pressure scenario 1: Tier 2 asked to dispatch Tier 3 without a spec."""
skill = _load_skill("mma-orchestrator")
assert_rule_present(
skill,
r"Surgical Spec Protocol",
msg="Surgical Spec Protocol section missing — Tier 2 cannot dispatch Tier 3 without it",
)
def test_pre_delegation_checkpoint_required(self):
"""Pressure scenario 2: Tier 2 delegates before staging/committing."""
skill = _load_skill("mma-orchestrator")
assert_rule_present(
skill,
r"Pre-Delegation Checkpoint",
msg="Pre-Delegation Checkpoint rule missing — worker failures without checkpoint lose prior iterations",
)
def test_persistent_tier2_memory_required(self):
"""Pressure scenario 3: Tier 2 applies Context Amnesia to itself."""
skill = _load_skill("mma-orchestrator")
assert_rule_present(
skill,
r"Persistent Tech Lead Memory|Persistent Tier 2|No Context Amnesia",
msg="Persistent Tier 2 memory rule missing — Tier 2 must maintain context throughout a track",
)
def test_failure_count_escalation_required(self):
"""Rule coverage: failure_count escalation pattern for repeated worker failures."""
skill = _load_skill("mma-orchestrator")
assert_rule_present(
skill,
r"failure-count|failure_count",
msg="failure_count escalation pattern missing — repeated worker failures must escalate to a more capable model",
)
def test_audit_documents_required(self):
"""Rule coverage: Architecture Fallback audit documents reference the 5 deep-dive guides."""
skill = _load_skill("mma-orchestrator")
assert_rule_present(
skill,
r"Architecture Fallback|guide_architecture\.md|guide_tools\.md|guide_mma\.md",
msg="Architecture Fallback audit docs missing — Tier 1 must consult docs/guide_*.md before track creation",
)
# ============================================================
# mma-tier1-orchestrator: product alignment + track initialization.
# ============================================================
class TestMmaTier1Orchestrator:
"""mma-tier1-orchestrator: product alignment + track initialization."""
def test_audit_before_specifying_required(self):
"""Pressure scenario 1: Tier 1 asked to write a spec without auditing the codebase."""
skill = _load_skill("mma-tier1-orchestrator")
assert_rule_present(
skill,
r"Audit|audit the codebase",
msg="Audit-before-specifying rule missing — specs without audit re-implement existing features",
)
def test_spec_gaps_not_features_required(self):
"""Pressure scenario 2: Tier 1 asked to add features instead of filling gaps."""
skill = _load_skill("mma-tier1-orchestrator")
assert_rule_present(
skill,
r"gaps, not features|GAPS, NOT FEATURES",
msg="Spec-gaps-not-features rule missing — features without gaps break the existing implementation",
)
def test_worker_ready_tasks_required(self):
"""Rule coverage: WHERE/WHAT/HOW/SAFETY per-task contract."""
skill = _load_skill("mma-tier1-orchestrator")
assert_rule_present(
skill,
r"WHERE.*WHAT.*HOW.*SAFETY|worker-ready",
msg="Worker-ready tasks rule missing — task descriptions must be executable by Tier 3 workers",
)
def test_root_cause_analysis_required(self):
"""Pressure scenario 3: Tier 1 spec for fix track lists no root cause candidates."""
skill = _load_skill("mma-tier1-orchestrator")
assert_rule_present(
skill,
r"[Rr]oot [Cc]ause",
msg="Root cause analysis rule missing — fix tracks without root cause candidates waste time",
)
def test_reference_docs_required(self):
"""Rule coverage: Reference docs (docs/guide_*.md) in every spec."""
skill = _load_skill("mma-tier1-orchestrator")
assert_rule_present(
skill,
r"Reference [Dd]ocs|guide_\*\\.md|guide_\*.md",
msg="Reference docs requirement missing — specs must link to existing docs/guide_*.md patterns",
)
# ============================================================
# mma-tier2-tech-lead: track execution + implementation oversight.
# ============================================================
class TestMmaTier2TechLead:
"""mma-tier2-tech-lead: track execution + implementation oversight."""
def test_atomic_per_task_commits_required(self):
"""Pressure scenario 1: Tier 2 batches 3 tasks into one commit for efficiency."""
skill = _load_skill("mma-tier2-tech-lead")
assert_rule_present(
skill,
r"ATOMIC PER-TASK|atomic.{0,20}per-task|Per-Task Atomic",
msg="Atomic per-task commits rule missing — batched commits defeat safe rollback",
)
def test_tdd_enforcement_required(self):
"""Pressure scenario 2: Tier 2 accepts worker code without a Red-phase test."""
skill = _load_skill("mma-tier2-tech-lead")
assert_rule_present(
skill,
r"TDD Enforcement|TDD enforcement|Red.{0,20}[Pp]hase|failing test",
msg="TDD enforcement rule missing — Tier 2 must verify the Red phase before delegating",
)
def test_persistent_context_required(self):
"""Rule coverage: Tier 2's persistent-context disclaimer."""
skill = _load_skill("mma-tier2-tech-lead")
assert_rule_present(
skill,
r"[Pp]ersistent context|No Context Amnesia|maintains persistent context",
msg="Persistent context rule missing — Tier 2 must NOT use Context Amnesia on its own session",
)
def test_anti_entropy_state_audit_required(self):
"""Pressure scenario 3: Tier 2 adds a new state variable without checking `__init__`."""
skill = _load_skill("mma-tier2-tech-lead")
assert_rule_present(
skill,
r"Anti-Entropy|state audit|__init__|existing, unused.{0,30}duplicate",
msg="Anti-entropy state audit rule missing — redundant state variables break the DOD discipline",
)
def test_surgical_delegation_prompts_required(self):
"""Rule coverage: Surgical delegation protocol (WHERE/WHAT/HOW/SAFETY)."""
skill = _load_skill("mma-tier2-tech-lead")
assert_rule_present(
skill,
r"Surgical Delegation|WHERE.*WHAT.*HOW.*SAFETY",
msg="Surgical delegation protocol missing — Tier 3 dispatch prompts must include file:line references",
)
# ============================================================
# mma-tier3-worker: TDD implementation + surgical code changes.
# ============================================================
class TestMmaTier3Worker:
"""mma-tier3-worker: TDD implementation + surgical code changes."""
def test_tdd_mandatory_enforcement_required(self):
"""Pressure scenario 1: Tier 3 writes code before writing the failing test."""
skill = _load_skill("mma-tier3-worker")
assert_rule_present(
skill,
r"TDD Mandatory Enforcement|failing test.{0,50}verify it fails|RED phase",
msg="TDD mandatory enforcement rule missing — Tier 3 writes test first, watches it fail, then writes code",
)
def test_no_architectural_decisions_required(self):
"""Pressure scenario 2: Tier 3 makes an architectural decision to 'improve' the code."""
skill = _load_skill("mma-tier3-worker")
assert_rule_present(
skill,
r"[Dd]o not make architectural decisions|architectural decisions.{0,30}Tier",
msg="No architectural decisions rule missing — Tier 3 must defer to Tier 2 on architecture",
)
def test_no_unrelated_file_modifications_required(self):
"""Pressure scenario 3: Tier 3 modifies files outside its task scope."""
skill = _load_skill("mma-tier3-worker")
assert_rule_present(
skill,
r"[Dd]o not modify unrelated files|unrelated files beyond the immediate task",
msg="No unrelated file modifications rule missing — Tier 3 scope is bounded",
)
def test_stateless_operation_required(self):
"""Rule coverage: Context Amnesia — Tier 3 is stateless per task."""
skill = _load_skill("mma-tier3-worker")
assert_rule_present(
skill,
r"stateless|Context Amnesia",
msg="Stateless operation rule missing — Tier 3 must assume each task starts with a clean context",
)
def test_skeleton_views_dependency_required(self):
"""Rule coverage: Skeleton Views from Tier 2/Orchestrator for dependency context."""
skill = _load_skill("mma-tier3-worker")
assert_rule_present(
skill,
r"Skeleton Views|skeleton.{0,20}views",
msg="Skeleton Views dependency rule missing — Tier 3 must rely on curated context, not raw file reads",
)
# ============================================================
# mma-tier4-qa: test analysis + error summarization + bug reproduction.
# ============================================================
class TestMmaTier4Qa:
"""mma-tier4-qa: test analysis + error summarization + bug reproduction."""
def test_stateless_operation_required(self):
"""Pressure scenario 1: Tier 4 carries context across multiple error analyses."""
skill = _load_skill("mma-tier4-qa")
assert_rule_present(
skill,
r"stateless|Context Amnesia",
msg="Stateless operation rule missing — Tier 4 must be stateless per analysis",
)
def test_no_fix_implementation_required(self):
"""Pressure scenario 2: Tier 4 implements the fix while diagnosing."""
skill = _load_skill("mma-tier4-qa")
assert_rule_present(
skill,
r"[Dd]o not implement the fix|implements? the fix|implementing the fix",
msg="No fix implementation rule missing — Tier 4 analyzes; Tier 2/Tier 3 implement",
)
def test_brief_output_required(self):
"""Pressure scenario 3: Tier 4 produces a 400-line trace summary."""
skill = _load_skill("mma-tier4-qa")
assert_rule_present(
skill,
r"brief|briefly|concise|brief.{0,20}summary",
msg="Brief output requirement missing — Tier 4 output must be concise (typical 20-word summary)",
)
def test_root_cause_analysis_required(self):
"""Rule coverage: Root cause analysis is Tier 4's primary deliverable."""
skill = _load_skill("mma-tier4-qa")
assert_rule_present(
skill,
r"[Rr]oot [Cc]ause",
msg="Root cause analysis rule missing — Tier 4 must identify root cause, not just symptoms",
)
def test_diagnostic_tools_required(self):
"""Rule coverage: Diagnostic and exploration tools usage."""
skill = _load_skill("mma-tier4-qa")
assert_rule_present(
skill,
r"[Dd]iagnostic|exploration tools|get_file_summary|get_code_outline",
msg="Diagnostic tools requirement missing — Tier 4 must use exploration tools, not raw file reads",
)