# Track Specification: Deep AST-Driven Context Pruning (deep_ast_context_pruning_20260306)

## Overview
Use tree_sitter to parse target file AST and inject condensed skeletons into worker prompts. Currently workers receive full file context; this track reduces token burn by injecting only relevant function/method signatures.

## Current State Audit

### Already Implemented (DO NOT re-implement)

#### ASTParser in file_cache.py (src/file_cache.py)
- **Uses tree-sitter** with `tree_sitter_python` language
- **`ASTParser.get_skeleton(code: str) -> str`**: Returns file with function bodies replaced by `...`
- **`ASTParser.get_curated_view(code: str) -> str`**: Enhanced skeleton preserving `@core_logic` and `# [HOT]` bodies
- **Pattern**: Parse → Walk AST → Identify function_definition nodes → Preserve signature/docstring, replace body

#### Worker Context Injection (multi_agent_conductor.py)
- **`run_worker_lifecycle()`** function handles context injection
- **First file**: Gets `get_curated_view()` (full hot paths)
- **Subsequent files**: Get `get_skeleton()` (signatures only)
- **`context_requirements`**: List of files from Ticket dataclass

#### MCP Tool Integration (mcp_client.py)
- **`py_get_skeleton()`**: Already exposes skeleton generation as tool
- **`py_get_code_outline()`**: Returns hierarchical outline with line ranges
- **Tools available to workers** for on-demand full reads

### Gaps to Fill (This Track's Scope)
- Workers still receive full first file in some cases
- No selective function extraction based on ticket target
- No caching of parsed ASTs (re-parse on each context build)
- Token reduction not measured/verified

## Architectural Constraints

### Parsing Performance
- AST parsing MUST complete in <100ms per file
- tree-sitter is already fast (C extension)
- Consider caching parsed trees in memory

### Skeleton Quality
- Must preserve enough context for worker to understand interface
- Must preserve docstrings for API documentation
- Must preserve type hints in signatures

### Worker Autonomy
- Workers MUST still be able to call `py_get_definition` for full source
- Skeleton is the default, not the only option
- Workers can request full reads on-demand

## Architecture Reference

### Key Integration Points

| File | Lines | Purpose |
|------|-------|---------|
| `src/file_cache.py` | 30-80 | `ASTParser` class with tree-sitter |
| `src/multi_agent_conductor.py` | 150-200 | `run_worker_lifecycle()` context injection |
| `src/models.py` | 30-50 | `Ticket.context_requirements` field |
| `src/mcp_client.py` | 200-250 | `py_get_skeleton()` MCP tool |

### tree-sitter Pattern (existing)
```python
from file_cache import ASTParser
parser = ASTParser("python")
tree = parser.parse(code)
skeleton = parser.get_skeleton(code)
curated = parser.get_curated_view(code)
```

## Functional Requirements

### FR1: Targeted Function Extraction
- Given a ticket's `target_file` and context, identify relevant functions
- Extract only those function signatures + docstrings
- Include imports and class definitions they depend on

### FR2: Dependency Graph Traversal
- For target function, find all called functions
- Include signatures of dependencies (not full bodies)
- Limit depth to prevent explosion

### FR3: AST Caching
- Cache parsed AST trees per file path
- Invalidate cache when file mtime changes
- Use `file_cache` pattern already in place

### FR4: Token Measurement
- Log token count before/after pruning
- Calculate reduction percentage
- Display in MMA dashboard or logs

## Non-Functional Requirements

| Requirement | Constraint |
|-------------|------------|
| Parse Time | <100ms per file |
| Memory | Cache size bounded (LRU, max 10 files) |
| Token Reduction | >50% for typical worker prompts |

## Testing Requirements

### Unit Tests
- Test targeted extraction returns only specified functions
- Test dependency traversal includes correct functions
- Test cache invalidation on file change

### Integration Tests
- Run worker with skeleton context, verify completion
- Compare token counts: full vs skeleton
- Verify worker can still call py_get_definition

### Performance Tests
- Measure parse time for files of various sizes
- Verify <100ms for files up to 1000 lines

## Out of Scope
- Non-Python file parsing (Python only for now)
- Cross-file dependency tracking
- Automatic relevance detection (manual target specification only)

## Acceptance Criteria
- [ ] Targeted function extraction works
- [ ] Token count reduced by >50% for typical prompts
- [ ] Workers complete tickets with skeleton-only context
- [ ] AST caching reduces re-parsing overhead
- [ ] Token reduction metrics logged
- [ ] >80% test coverage for new code
- [ ] 1-space indentation maintained
