The user said (verbatim): "On number 1. I love the idea and definitely
see poitental." This commit creates a full track that promotes the
ASCII-sketch UX ideation workflow
(docs/reports/ascii_sketch_ux_workflow_20260608.md, 340 lines) to
a real track with a concrete first target.
The track complements (does not replace) the existing
manual_ux_validation_20260302 track (which is a general UX review
track; this 2026-06-08 track is *focused* on the ASCII-sketch
workflow specifically).
Files (5 total, ~52KB, 12,000+ words):
- spec.md (186 lines, 9 sections) - track design, 5 open
questions, first target analysis, SSDL cross-reference
- plan.md (~280 lines, 4 phases, 21 tasks) - TDD-style with
WHERE/WHAT/HOW/SAFETY annotations
- metadata.json (~120 lines) - structured metadata, 5 open
questions with defaults, 5 SSDL principles available
- state.toml (~95 lines) - per-task tracking + phase status
- index.md (~50 lines) - track context + related docs
Key design decisions captured:
1. Two distinct vocabularies are conflated at first glance:
- GUI ASCII (the workflow) for panel sketches
- SSDL (computational shapes digest) for internal code sketches
Spec §2.6 makes the distinction explicit; both are useful for
this track (GUI ASCII for Phase 2 design; SSDL for Phase 3
internal refactoring documentation).
2. The 5 open questions from the workflow report (Q1 vocabulary,
Q2 comparison policy, Q3 storage location, Q4 tooling,
Q5 frequency) are documented with sensible defaults in
spec.md §2.1-2.5 and metadata.json. The user can override
any of them; defaults pre-stage the work.
3. First target is src/gui_2.py:3770 render_discussion_entry
(Discussion Hub per-entry panel). Rationale:
- Most-edited surface (every AI/user message)
- User has strong opinions (per nagent_review_20260608 3 rounds
of corrections)
- 23-op matrix A1-A7 is the source of truth
- ImGui layout maps cleanly to ASCII
- SSDL defusing techniques can guide the internal refactoring
4. 4 phases: 1=resolve 5 questions, 2=execute workflow on first
target (1-3 ASCII rounds), 3=implement per design contract
(TDD with 7 test files for A1-A7 operations),
4=document the pattern + propose 5-7 next targets.
Cross-references added throughout:
- docs/reports/computational_shapes_ssdl_digest_20260608.md
(the SSDL digest, with explicit "this is a different vocabulary
for a different purpose" note in spec §2.6)
- docs/reports/ascii_sketch_ux_workflow_20260608.md (the workflow)
- docs/guide_discussions.md (the 23-op matrix A1-A7)
- conductor/tracks/nagent_review_20260608/ (the source of the
user's editable-discussion corrections)
- conductor/tracks/manual_ux_validation_20260302/ (complementary
general UX review track)
- conductor/tracks/chunkification_optimization_20260608_PLACEHOLDER/
(the contingency track; referenced in spec §2.6 SSDL cross-ref)
No code modified. Track is active; Phase 1 (5 user-questions) is
the current phase. User-confirmed worth doing in the prior turn.
The user's third correction this session changed the framing
from "build a stateful C extension" to "wait for a hard constraint,
then build a request/response blob pipeline." This commit creates
a 1-page contingency document (no plan.md, no implementation)
that captures:
- The threshold: "only worth it under a hard constraint that
no existing Python package can solve"
- The shape when activated: subprocess-launch C11 binary with
request/response blob wire format (NOT stateful CPython C
extension)
- The 2 cited candidates (markdown parsing into aggregate markdown,
context snapshot processing) are NOT currently bottlenecks per
src/aggregate.py:380-454 (pure-Python string concat, zero
third-party markdown deps in pyproject.toml:6-27) and
src/history.py:1-141 (bounded ~500KB at 100-snapshot capacity,
debounced)
- The SSDL digest's Technique 5 "Assume-away (Xar)" in §2.2 +
"Xar-style chunked arrays" recommendation in §5.2 pre-support
this track
Files (4 total, 227+ lines of contingency document):
- conductor/tracks/chunkification_optimization_20260608_PLACEHOLDER/spec.md
- conductor/tracks/chunkification_optimization_20260608_PLACEHOLDER/metadata.json
- conductor/tracks/chunkification_optimization_20260608_PLACEHOLDER/state.toml
- conductor/tracks/chunkification_optimization_20260608_PLACEHOLDER/index.md
Cross-references added:
- docs/reports/computational_shapes_ssdl_digest_20260608.md (the
SSDL digest is the theoretical foundation; explicitly cited in
the spec's §6.1 "SSDL alignment" and in metadata.json external)
- docs/reports/c11_python_interop_assessment_20260608.md (the v1+v2
assessment; explicitly cited in spec's §6 See Also)
No code modified. Track does NOT appear in the active queue
of conductor/tracks.md; appears in the Backlog / Contingency
section as a reference, not a commitment.
Activation criteria (per metadata.json):
1. Profiling shows a real bottleneck in a target code path
2. The bottleneck cannot be solved with existing Python packages
3. The user explicitly approves activation
Without all 3, this track stays deferred. Default action is don't.
The user specified that the code_path_audit_20260607 track should run
AFTER the 4 foundational tracks complete (qwen_llama_grok,
data_oriented_error_handling, data_structure_strengthening,
mcp_architecture_refactor). This commit formalizes that timing
and grounds the audit's analytical framing in the 5 sources loaded
into context on 2026-06-08.
3 surgical additions to the spec/plan, no task changes:
1. Post-4-tracks timing (new section in spec.md §"Timing", plus
a "Timing" callout in plan.md's opening):
- The 4 tracks will significantly reshape src/ai_client.py,
src/mcp_client.py, src/app_controller.py, and
src/type_aliases.py
- Running the audit on pre-refactor code would produce a
report that's stale on day 1
- The post-4-tracks timing ensures the audit grounds
optimization decisions for the *resulting* architecture
- Pre-flight check: verify all 4 tracks are [x] completed
in conductor/tracks.md before starting this track
2. Analytical framing (new section in spec.md §"Analytical Framing
(5-source lens)"):
- Maps each of the 5 sources (Fleury taxonomy + Fleury
combinatoric + Muratori Big OOPs + Reece Assuming + user's
chunk ideation) to specific audit-time heuristics
- 4 concrete heuristics: effective-codepath count,
entity-hierarchy fingerprint, assumed-too-much detector,
chunkification candidates
- The heuristics shape REPORT INTERPRETATION, not the
static cost model (which stays data-grounded in
EXPENSIVE_THRESHOLD + per-class weights)
3. See Also cross-references in spec.md (6 new entries):
- nagent_review Pitfalls #2 and #4 (provider history
globals + stateful singleton)
- wo84LFzx5nI Big OOPs transcript (full text, 4310
segments, 200KB; loaded 2026-06-08)
- i-h95QIGchY Assuming transcript (full text, 3719
segments, 162KB; loaded 2026-06-08)
- ed_chunk_data_structures_20260523.md (5-image archive
of user's chunk ideation, 19KB; saved 2026-06-08)
- computational_shapes_ssdl_digest_20260608.md (the SSDL
digest that synthesizes the 4-source computational-shapes
thinking; the audit's tree/mermaid outputs ARE
computational-shape visualizations)
4. tracks.md entry updated to include the spec/plan links and
a brief status note that the audit is post-4-tracks.
5. plan.md has a "Timing" callout at the top stating the 4
tracks must ship before the plan executes.
No code modified. The audit's tasks (Phases 1-6) are unchanged
in structure; the new sections only add analytical context
and timing constraints.
4 surgical additions to the spec, no task changes:
1. list_tool_schemas on the SubMCP Protocol: Added the method
to §3.1 (The SubMCP Protocol). Per nagent_review Pitfall #6
(hard-coded tool discovery) and takeaway #5 (self-describing
tools), each sub-MCP advertises its own capabilities via
list_tool_schemas() rather than relying on a central registry.
This is the equivalent of nagent's collect_bin_tool_descriptions
per sub-MCP. The MCPController.get_tool_schemas() becomes a
simple aggregator.
2. Security model is the contract: Added a new Important note
to §3.3 (The 3-Layer Security Model). The 3 layers
(Allowlist Construction -> Path Validation -> Resolution
Gate, per docs/guide_mcp_client.md) are not just refactored
- they are the CONTRACT between MCPController and the
sub-MCPs. Sub-MCPs receive a pre-validated Path and trust
it. They do NOT re-validate. The refactor is structural,
not security-changing.
3. Docs touchpoint in Phase 7: Added the docs touchpoint to
Phase 7 per the docs Refresh Protocol. The update to
docs/guide_mcp_client.md should add a Sub-MCP Architecture
section, link the list_tool_schemas pattern to 3-Layer
Security Model, and cross-link the 3 new guides from
the 2026-06-08 docs refresh.
4. See Also cross-references: Added 8 new entries to §12.2:
- docs/guide_context_aggregation.md (FileItem consumer)
- docs/guide_state_lifecycle.md (App state delegation)
- docs/guide_discussions.md (23-operation matrix)
- conductor/tracks/qwen_llama_grok_integration_20260606/
(Result return type coordination)
- conductor/tracks/nagent_review_20260608/{report,takeaways}.md
- (2 specific data_oriented_error_handling and
data_structure_strengthening cross-refs)
No plan.md changes.
4 surgical additions to the spec, no task changes:
1. ProviderHistoryMessage: Added a new alias to §3.1 (The
Aliases). Per nagent_review Pitfall #4 (provider history
divergence), the UI/curation layer (HistoryMessage, edited
via disc_entries[i].content) and the SDK layer
(ProviderHistoryMessage, the bytes actually replayed to the
LLM) are *distinct*. Conflating them via a single alias
perpetuates the bug. The new alias is documented as a
separate concept with its own use sites (_anthropic_history,
_deepseek_history, _minimax_history, _grok_history,
_llama_history). The follow-up public_api_migration_20260606
track is the natural moment to unify the two layers; this
spec just makes the distinction explicit.
2. FileItem alias points to the existing models.FileItem
dataclass, not Metadata. Per docs/guide_context_aggregation.md
(added 2026-06-08), FileItem is a 9-field dataclass
(path, auto_aggregate, force_full, view_mode, selected,
ast_signatures, ast_definitions, ast_mask, custom_slices,
injected_at) with a __post_init__ normalizer. Aliasing it to
dict[str, Any] would lose the type safety. The 9 other
aliases remain dict aliases for round-trip compatibility.
3. gui_2.py and mcp_client.py as follow-up: Added a Note
(dated 2026-06-08) to the Out of Scope section. The 23
lower-impact files (deferred) are dominated by gui_2.py
(26+ weak sites per guide_state_lifecycle.md) and
mcp_client.py (will be touched heavily by the parallel
mcp_architecture_refactor_20260606). The deferral is correct
but the follow-up should explicitly call out these two
files as the next targets, rather than implying they're
handled.
4. See Also cross-references: Added 7 new entries to §12.2:
- docs/guide_models.md (FileItem dataclass source)
- docs/guide_context_aggregation.md (FileItems consumer)
- docs/guide_discussions.md (HistoryMessage shape)
- docs/guide_state_lifecycle.md (state delegation)
- conductor/tracks/mcp_architecture_refactor_20260606/
- conductor/tracks/nagent_review_20260608/{report,takeaways}.md
No plan.md changes.
3 surgical additions to the spec, no task changes:
1. New ErrorKind: Added PROVIDER_HISTORY_DIVERGED_FROM_UI to
the ErrorKind enum. Per nagent_review Pitfall #4 (provider
history divergence: user edits disc_entries[i].content via
the discussion UI but ai_client._<provider>_history still
replays the original). The new kind makes the divergence
*detectable* and *reportable* so the follow-up
public_api_migration_20260606 track can collapse the two
history layers. The Result pattern from this track is the
natural carrier for the signal.
2. State-delegation regression tests: Added mandatory
regression tests to the testing strategy in §6 for the
ai_client refactor (highest-risk phase). The new tests
exercise:
- app.temperature = 0.5 round-trips through App.__getattr__/
__setattr__ delegation (per gui_2.py:666-675)
- controller.disc_entries[i].content is reflected in the
next send_result()'s messages parameter
- The 3 per-provider history locks serialize correctly under
concurrent send_result() calls
The reason this is mandatory: per guide_state_lifecycle.md
(added 2026-06-08), the App.__getattr__/__setattr__ pattern
means a partial refactor manifests as silent AttributeError
deep in test code, not at the refactor commit boundary.
3. See Also cross-references: Added 6 new entries to §12.3:
- docs/guide_ai_client.md (per-provider history globals)
- docs/guide_mcp_client.md (3-layer security model)
- docs/guide_state_lifecycle.md (3 per-thread + 7-lock pattern)
- docs/guide_discussions.md (23-operation matrix)
- docs/guide_context_aggregation.md (build_discussion_section)
- conductor/tracks/mcp_architecture_refactor_20260606/
- conductor/tracks/nagent_review_20260608/{report,takeaways}.md
No plan.md changes. Plan tasks are task-level and will flow from
the spec changes when the track is re-planned.
4 surgical additions to the spec, no task changes:
1. Result return type: Added a coordination note in §3.1 (Data-
Oriented Design) explaining that the shared send_openai_compatible
helper should return Result[NormalizedResponse, ErrorInfo] from
day 1, not NormalizedResponse + ProviderError raise. This is so
the downstream data_oriented_error_handling_20260606 track is
a small mechanical pass over new code, not a second migration.
References nagent_review Pitfall #4 (provider history divergence)
and the ErrorKind.PROVIDER_HISTORY_DIVERGED_FROM_UI use case.
2. Declarative read, not behavioral dispatch: Added clarification
to §6 (UX Adaptation) that the capability matrix is a *read* of
declarative data, not a new dispatch layer. Per nagent_review
Pitfall #1 (opaque function calling in the Application is the
correct choice; nagent-style protocol is for Meta-Tooling),
UI elements are visible/enabled/disabled/hidden but the
*behavior* they invoke is unchanged. Three concrete examples
added: screenshot button, cost panel, cache panel.
3. PROVIDERS source of truth: Added a NOTE in §3.2 (Module Layout)
that src/models.py:79-86 PROVIDERS is the existing single
source of truth for the (vendor, model) enumeration. The
capability registry reads from this constant rather than
introducing a parallel list. Cross-references
docs/guide_models.md.
4. Docs touchpoint: Expanded Phase 6 (Docs + Archive) in §9 to
note that docs/guide_ai_client.md needs the new providers +
the shared helper documented, and that
docs/guide_context_aggregation.md (added 2026-06-08) is the
reference for the aggregate.py pipeline that all new providers
use.
5. See Also cross-references: Added 3 new entries to §13.2:
- docs/guide_context_aggregation.md (the new pipeline guide)
- conductor/tracks/nagent_review_20260608/report.md (§1, §5, §15)
- conductor/tracks/nagent_review_20260608/nagent_takeaways_20260608.md
(§1, §2, §9)
No plan.md changes. Plan tasks are task-level and will flow from
the spec changes when the track is re-planned.
Reference/analysis track. Produces 0 code changes.
Artifacts (conductor/tracks/nagent_review_20260608/):
- spec.md (240 lines) - track wrapper with Application/Meta-Tooling framing
- report.md (571 lines) - 14-section deep-dive; primary deliverable
- comparison_table.md (79 lines) - flat side-by-side reference
- decisions.md (286 lines) - 10 future-track candidates with priority matrix
- nagent_takeaways_20260608.md (363 lines) - 10 actionable patterns grounded
in code (file:line refs into nagent source and Manual Slop source)
- metadata.json (132 lines) - structured metadata + verification criteria
- state.toml (113 lines) - per-task tracking + user-corrections log (7 entries)
14 nagent principles covered in report.md (durable work, text-in/text-out,
editable state, visible protocol, the loop, per-file memory, repo history,
neighborhoods, sub-conversations, controlled writes, large files, tool
discovery, framework differences, build your own).
6 pitfalls (revised from 8 after user-corrections):
1. No structured output protocol in Application AI (opaque function calling)
2. Provider-specific history in process globals (ai_client._anthropic_history
+ _deepseek_history + _minimax_history)
3. RAG is not 'history as data' (fuzzy, not auditable)
4. AI client is a stateful singleton (2,685-line ai_client.py)
5. No non-MMA disposable sub-conversations (1:1 gap; user-flagged want)
6. Hard-coded tool discovery (45-tool if/elif in mcp_client.py)
User-corrections applied (3 rounds, 7 total corrections recorded):
- Editable discussions: PARTIAL -> PARITY (DIFFERENT FOCUS) with full A1-A7
per-entry + B1-B11 discussion-level + C1-C5 undo/redo operation matrix
- Per-file memory: DOMAIN MISMATCH -> MANUAL SLOP IS STRONGER IN
CURATION DIMENSION (FileItem + ContextPreset vs nagent's inode-keyed
conversation log; complementary, not equivalent)
- Sub-conversations: MMA has it; 1:1 does not -> 'PARITY for MMA; GAP for
1:1 discussions' (user wants this)
- RAG: opt-in, not gap; user wants pre-staging via sub-conversation
- Personas: config bundling (can opt out via AI settings)
- Tool discovery: deferred (user has 'intent based DSL' idea but 'no where
near that ideation yet')
10 actionable takeaways (separate from the 6 pitfalls - those are
diagnosis, these are prescription):
1. State visibility (UI inspector for in-process state)
2. Readable conversation log (text-greppable, not just JSON-L)
3. Sub-agents for 1:1 (HIGH priority - user-flagged)
4. File-identity over file-path (st_dev:st_ino rename-safe)
5. One loop shape visible in diagnostics
6. Visible retry on protocol failure
7. Meta-Tooling DSL (intent-based, deferred)
8. Self-describing tools (subsumed by mcp_architecture_refactor_20260606)
9. Single source of truth for disc_entries + provider history
10. Sub-agent return type constraint (bake into candidate #1 spec)
Domain classification: every recommendation tagged Application / Meta-Tooling
/ Both per docs/guide_meta_boundary.md. nagent lives in the Meta-Tooling
domain; Manual Slop's Application AI is a different kind of thing.
No code modified by this track (reference/analysis only). All 7 files
parse cleanly (JSON, TOML, Markdown). All internal cross-links resolve.
Track is 'active' awaiting human review; future-track candidates live in
decisions.md and nagent_takeaways_20260608.md.
Phase 4 verification complete: 4 atomic commits landed, 28
unit + integration tests passing, the audit script runs
end-to-end against the post-cleanup repo, --strict mode
+ baseline file wired in as the CI gate. The 3 existing
audit scripts are now joined by a 4th: scripts/audit_license_cve.py.
Scope: third-party deps only. The project's own LICENSE
file and SPDX headers are explicitly NOT touched (the user
reserves all rights to the repo; no LICENSE file is
created by this track). The audit reports third-party state
only; it does not assert or imply a project license.
Commits:
a8ae11d3 - chore(audit): add license_cve audit script + initial report
20fa3558 - chore(deps): tilde-pin all deps; delete requirements.txt
a7ab994f - chore(audit): add --strict mode + baseline file (CI gate)
(this) - conductor(tracks): mark track complete
scripts/audit_license_cve.py: 4 internal checks (license +
CVE + pin + source-header), policy tables (allowlist of
permissive/weak-copyleft/public-domain, blocklist of
non-OSI/restricted-source), and a main() that runs all 4
and emits line-per-violation to stdout + a markdown report.
Tests (26 unit + integration) cover license classifier (16
variants across MIT, BSD, Apache, LGPL, MPL, CC0, WTFPL,
GPL, AGPL, SSPL, BSL, Commons Clause, Elastic, Anti-996,
Hippocratic, unknown), pin check (3), source-header check
(3), license check via importlib.metadata (1), CVE check
via subprocess pip-audit (2), and a smoke test of the main
loop (1).
No new pip deps in the project: pure stdlib
(importlib.metadata, tomllib, pathlib, re) + subprocess to
pip-audit (optional dev tool, installed via 'uv tool install
pip-audit' if user wants CVE checks).
Initial report at docs/reports/license_cve_audit/2026-06-07/
records the current state. The Phase 2 commit will apply
the fixes (tilde-pin, delete requirements.txt); the Phase 3
commit will add --strict mode + baseline file for CI.
The _push_mma_state_update method (added in 8216d494) used
models.TicketState for the persisted tasks list, but:
- src.models has no TicketState class; only Ticket
- TrackState.tasks is annotated as List[Ticket]
So my code raised AttributeError on every call, which my
try/except caught and silently printed. Tests that depended
on save_track_state being called (test_push_mma_state_update)
failed because the call was skipped.
Also fixed:
- TrackState field name: it's 'tasks' (not 'tickets') per the
src.models dataclass annotation. My code was using 'tickets='
which created a TypeError on construction.
- Removed the [DEBUG ...] print statements added during the
investigation; they were only for diagnosing the silent
AttributeError.
- Kept the try/except so a real exception is still logged to
stderr (visible via -s flag) without breaking the test.
Result: 11/11 tests in test_gui_phase4 + test_ticket_queue now
pass:
- test_push_mma_state_update
- test_ticket_priority_default/custom/to_dict/from_dict
- TestBulkOperations::test_bulk_execute/skip/block (3)
- TestReorder::test_reorder_ticket_valid/invalid (2)
Builds scripts/audit_license_cve.py: single audit script that
checks third-party deps (pyproject.toml + uv.lock transitive
tree) for: (1) license compliance against the project's policy,
(2) known CVEs (via pip-audit subprocess), (3) version-pinning,
and (4) source-file SPDX license headers in src/ and scripts/.
LICENSE POLICY (encoded in the script)
Allowlist (permissive or weak copyleft or public domain):
- Permissive: MIT, BSD, Apache 2.0, ISC, Unlicense, Zlib,
Python-2.0, 0BSD, PSF-2.0
- Weak copyleft (Python import-safe): LGPL 2.1/3.0, MPL-2.0
- Public domain: CC0, WTFPL
Blocklist (non-OSI / restricted-source):
- GPL (any version), AGPL (any version)
- SSPL (MongoDB 2018) - broad service-provider trigger
- BSL / BUSL - delayed open source; competitive-use restriction
- Commons Clause - 'cannot sell the software' addendum
- Elastic License v2 - 'cannot offer as managed service'
- Unknown / unparseable / missing metadata (catches packaging
bugs and custom licenses)
The two lists are explicit. Default rule: unknown = violation
(never auto-pass). The script's --help references the policy
table for transparency. Specific per-license additions go in
scripts/audit_license_cve.py directly; no spec change needed.
TRACK SCOPE
In scope: third-party deps (direct + transitive), source-file
SPDX headers, vendored libraries (defensive), version pinning.
Out of scope: the project's own LICENSE file, project's own
SPDX/Copyright headers, recommendations on project license.
The user reserves all rights to the repo; no LICENSE file is
created by the track. The audit reports third-party state only.
OUTPUT FORMAT (sanitized: no JSON in user-facing output)
- Stdout: line-per-violation, parseable by eye and by grep
- Markdown report in docs/reports/license_cve_audit/2026-06-07/
- Baseline file: JSON (matches existing audit_weak_types
convention; internal state for --strict mode only)
CI GATE
--strict mode + scripts/audit_license_cve.baseline.json. Fails
CI on any new violation OR any new CVE. Mirrors the 3 existing
audit scripts (audit_main_thread_imports, audit_weak_types,
check_test_toml_paths).
COMMITS PLANNED
1. chore(audit): add license_cve audit script + initial report
2. chore(deps): tilde-pin all deps; delete requirements.txt
3. chore(audit): add --strict mode + baseline file (CI gate)
4. conductor(tracks): mark License CVE Audit track complete
NO NEW PIP DEPENDENCIES IN PROJECT
Pure stdlib (importlib.metadata, tomllib, pathlib, re) +
subprocess to pip-audit (an optional dev tool, installed via
'uv tool install pip-audit' if user wants CVE checks).
Per user direction ('make a custom DSL ideal for recording the
call-graph or other metrics', 'I want a post-fix heiarchy', 'JSON
is ill-performant'): replaced JSON serializer with a custom
postfix (RPN) DSL tailored to the audit's record shapes.
THE CUSTOM DSL
- Postfix (operands before operator); no brackets, braces,
commas, or colons.
- Length-prefixed lists: N items followed by 'list' word.
- Tagged records: each 'word' is a constructor with a known
arity (action=3, fn=3, call=1, mut=3, exp-op=5, pair=2, int=1).
- Whitespace-tokenized; bare atoms unquoted; double quotes
only when whitespace/special chars present.
- nil for null; backslash for line comments; true/false for bool.
- Trivial parser (~30 lines): _tokenize_dsl splits on
whitespace and respects quotes + comments; parse_dsl
walks tokens and evaluates tagged words against a known
arity table (DSL_WORD_ARITY).
- Round-trips: to_dsl(profile) -> parse_dsl(to_dsl(profile))
yields the same in-memory structure.
DELIVERABLES (updated spec + plan)
- src/code_path_audit.py: to_dsl, dump_dsl, parse_dsl,
_tokenize_dsl, to_tree (prefix-tree text renderer),
to_markdown, to_mermaid.
- Output: .dsl files (machine) + .tree (human prefix view) +
.md (summary tables) + .mmd (Mermaid diagrams).
- No new pip dependencies; pure stdlib.
WHAT STAYED
- The 7 cost classes (file_io, network, ast_parse, json_io,
pickle, deep_copy, loop_amplified) and 5 mutation kinds
are unchanged. The json_io cost class is for JSON file
I/O the audit detects, not the output format.
- 36 tests total (15 + 8 + 10 + 3 across the 4 implementation
phases).
6 phases, one per commit:
Phase 1: data structures (CallGraph, ExpensiveOp, StateMutation)
- 15 unit tests
Phase 2: trace_action + ActionProfile + cost model + AST walking
- 8 tests (synthetic + integration on real src/)
Phase 3: JSON / markdown / Mermaid output
- 4 tests
Phase 4: MCP tool + CLI surface
- 3 tests
Phase 5: run audit on 3 actions; commit report
Phase 6: tracks.md update
TDD pattern: each task has synthetic-data unit test, then
real implementation, then integration with real src/, then
commit. The state.toml scaffold is created in Phase 0 Step 0.1
and advanced after each phase.
3 actions in scope (MMA is cold per user):
- ai_message_lifecycle (5 entry points)
- discussion_save_load (4 entry points)
- gui_startup (3 entry points)
Two follow-up tracks recorded but NOT in this track:
- pipeline_runtime_profiling_20260607
- pipeline_pruning_20260607
No new pip dependencies; pure stdlib (ast, json, pathlib,
dataclasses). Read-only on src/; new files are the tool, the
tests, and the report under docs/reports/code_path_audit/2026-06-07/.
Design for a data-oriented static-analysis tool
(src/code_path_audit.py) that audits the 3 major actions (AI
message lifecycle, discussion save/load, GUI startup) for
expensive operations, redundant calls, and pipelining
candidates. Output: JSON data files + markdown summaries +
Mermaid per-action call graphs in docs/reports/code_path_audit/.
61 src/ files, 27,447 total lines. Call graph is non-trivial;
per-action traversal is what makes analysis tractable.
Cost model: 7 cost classes (file_io, network, ast_parse,
json_io, pickle, deep_copy, loop_amplified) with heuristic
weights; EXPENSIVE_THRESHOLD = 40,000 module constant. 5
state mutation kinds (attr_write, container_mutate, file_write,
ipc_emit, global_write).
The 3 action entry points are per-action defined (see Per-Action
Design table). MMA worker spawn is OUT of scope per user (cold
until 1:1 discussion UX is dogfooded).
Two follow-up tracks recorded but NOT in this track:
- pipeline_runtime_profiling_20260607: calibrate the heuristic
cost model with real measurements; catch C-extension cost,
decorator dispatch, JIT effects that static analysis can't
resolve.
- pipeline_pruning_20260607: implement the high-priority
optimization candidates surfaced by this track's report.
6 atomic commits planned: data structures; trace_action +
ActionProfile + cost model; output (JSON/MD/Mermaid); MCP +
CLI; run audit + commit report; tracks.md update.
All 6 sub-tracks (2A-2F) complete. Audit script: 0 violations (was 67 baseline / 61 before sub-track 2). Track is now FULLY COMPLETE (was previously [~] due to sub-track 2 partial). 79 tests added/passing across sub-tracks 2A-2F. Updated sub_tracks table in state.toml with per-sub-track completion details. Pre-existing test failures (4 unrelated) documented in test_failure_notes.
5 phases, one per deletion category from the spec:
Phase 1: Remove one-shot indent fixers (10 files)
Phase 2: Remove one-shot transform scripts (6 files)
Phase 3: Remove superseded entropy and code-stat audits (4 files)
Phase 4: Remove one-shot migrators and repros (6 files)
Phase 5: Remove tool-call aliases and legacy tool discovery (4 files)
Phase 6: Final verification + tracks.md update
Each phase = one git rm + one commit + one git note + one
state.toml update. Phase 0 adds the state.toml scaffold. Phase 6
runs the full test suite in 4-at-a-time batches per workflow.md
Phase Completion protocol, re-runs the 2 active audit scripts
(main_thread_imports, weak_types) for regression check, and
commits the tracks.md update.
TDD pattern adapted for deletion: pre-deletion baseline (Phase 0)
+ per-phase git rm + post-deletion test suite pass (Phase 6).
No new code, no new tests, no new CI gate.
Design for removing 30 confirmed-unused one-off scripts from
scripts/. Net effect: scripts/ shrinks from 56 -> 26 files
(54% reduction). All deletions are hard deletes via 5 atomic
per-category commits; git log is the restore path.
26 KEEPS documented by category (CI gates, MMA, MCP, test runner,
ImGui linter, audit/scaffolding, tool-call bridge, Docker, borderline
utility). 30 DELETES grouped by category: one-shot indent fixers
(10), one-shot transform scripts (6), superseded entropy audits (4),
one-shot migrators/repros (6), tool-call aliases and legacy tool
discovery (4).
No new CI gate added. Follow-up unused_scripts_audit_20260607
recorded in the spec. Plan (writing-plans) will produce 5 phases
(one per category).
~25 tasks across 7 phases, each with explicit Red-Green-Refactor TDD steps:
- Phase 1 (1.1-1.5): Foundation. 3-layer security module (8 unit tests
returning Result[Path]); SubMCP Protocol + MCPController class (6 unit
tests). Controller added ALONGSIDE the existing 45 functions in
mcp_client.py (no removal yet).
- Phase 2 (2.1-2.4): Backward compat. git mv mcp_client.py to
mcp_client_legacy.py; create new mcp_client.py as a slim shim
re-exporting 45+ old symbols. 12 legacy shim tests verify the surface.
The 4 existing test files + src/app_controller.py:61 still work.
- Phase 3 (3.1-3.4): FileIOMCP extracted (9 tools, 10 unit tests).
- Phase 4 (4.1-4.4): PythonMCP extracted (14 tools, 14 unit tests).
- Phase 5 (5.1-5.5): CMCP, CppMCP, WebMCP, AnalysisMCP extracted
(4 sub-MCPs, 18 unit tests; pattern mirrors Phase 3/4).
- Phase 6 (6.1-6.3): ExternalMCP extracted from mcp_client_legacy.
Class name preserved (ExternalMCPManager).
- Phase 7 (7.1-7.5): Update dispatch() in the legacy shim to use the
new controller (inverted-dict O(1) lookup); update docs; manual
smoke test; archive the track.
Each sub-MCP follows the same template (class with name / description
/ tools / invoke; security check for path-taking tools; Result wrapping
in invoke(); delegation to legacy functions for the actual implementation).
The sub-MCPs are thin adapters in v1; a future track can move the
implementations into the sub-MCP files directly.
Self-review at the end maps every spec section to a task (no gaps),
confirms zero placeholders, and verifies type/method-name consistency
across phases (SubMCP Protocol, MCPController class, Result[str,
ErrorInfo], _resolve_and_check all defined in Phase 1; used
consistently across Phases 3-6).
Track + metadata + state + tracks.md registration for the 2,205-line
mcp_client.py split into a slim controller + 6 native sub-MCPs + 1
external sub-MCP.
Key design decisions (per user feedback):
- Naming convention: mcp_<type>.py for native MCPs (mcp_file_io.py,
mcp_python.py, mcp_c.py, mcp_cpp.py, mcp_web.py, mcp_analysis.py).
- ExternalMCPManager class name preserved (moves to mcp_external.py).
- Sub-MCP shape: class with name / description / tools / invoke().
- MCPController: holds ALL_SUB_MCPS list, inverted-dict tool lookup,
3-layer security (extracted to mcp_client_security.py), schema
aggregation.
- Each invoke() returns Result[str, ErrorInfo] (from
data_oriented_error_handling_20260606).
- Backward compat: mcp_client_legacy.py re-exports all 45+ old
symbols; the 4 existing test files + src/app_controller.py:61
direct call continue to work.
DSL future (per user notes on APL/K/Cosy): NOT in this track.
Documented in spec §12.1 as the mcp_dsl_20260606 follow-up.
Sub-MCP architecture is the natural unit to pair with a DSL emitter.
7 phases. ~22 task slots. New tests: 9 (one per sub-MCP + controller +
security + legacy). Modified tests: 4 (existing mcp_* tests must
pass unchanged).
Blocked by: data_oriented_error_handling_20260606, data_structure_strengthening_20260606.
Blocks: mcp_dsl_20260606 (future DSL track).
~22 tasks across 2 phases, each with explicit Red-Green-Refactor TDD steps:
- Phase 1 (1.1-1.12): Foundation. type_aliases.py (10 TypeAliases + 1
NamedTuple) with 8 unit tests. Mechanical replacement of 345 weak
sites in 6 files (ai_client 139, app_controller 86, models 51,
api_hook_client 32, project_manager 20, aggregate 17). Each file
has a per-substitution table for the mechanical replacement. Audit
script gains --strict mode + baseline file (CI gate). 4 audit tests.
- Phase 2 (2.1-2.10): FileItemsDiff NamedTuple integrated.
generate_type_registry.py (AST-based; 3 modes: default, --check,
--diff). Initial registry generated in docs/type_registry/ (8+ .md
files). 6 generator tests. Type aliases styleguide + product-guidelines
updates. Manual smoke test. Track archived.
The type registry generator uses --check mode for CI: it regenerates to
a temp dir and diffs against the committed registry; exit 1 if drift.
The agent's track-completion workflow is: regenerate -> review diff ->
commit. CI enforces --check on every PR.
Self-review at the end maps every spec section to a task (no gaps),
confirms zero placeholders, and verifies type/method-name consistency
across phases (all 10 aliases + FileItemsDiff defined in Task 1.2; used
consistently in Tasks 1.3-1.8 and Phase 2).
Per user feedback (2026-06-06): instead of a follow-up 'TypedDict
Migration' track, add a NEW deliverable: an auto-generated type registry
in docs/type_registry/ that captures the field information in docs form.
New files:
- scripts/generate_type_registry.py (NEW): AST-based tool that reads
src/ and writes per-source-file .md files with the fields of every
@dataclass, NamedTuple, TypeAlias, TypedDict. Has --check (CI mode,
exits 1 if registry would change) and --diff (dry run) modes.
- docs/type_registry/ (NEW, generated): index.md + per-source-file
references (type_aliases.md, ai_client.md, models.md, etc.).
- tests/test_generate_type_registry.py (NEW): verify the generator.
Architecture updates:
- Section 3.6 (NEW): Type Registry architecture with example output.
- Section 3.7 (NEW): Why per-source-file docs (locality of reference).
- Section 1.1 (NEW): 'Why docs over TypedDict' analysis (3 reasons:
lower upfront cost, better fit for AI workflow, auto-maintained).
- Goals table: registry added as a C (innovation) goal.
- Module layout: docs/type_registry/ and scripts/generate_type_registry.py
added to the new files list.
- Migration: Phase 2 now includes the registry generator + initial docs.
- Out of scope: TypedDict migration REMOVED; 'auto-typing the field
shape' added with the docs as the chosen approach.
- See Also: TypedDict follow-up REPLACED with 'Registry Maintenance &
CI Integration' (smaller scope, just wires the generator into CI).
The 'cost we eat' is the LLM reading 200-500 lines of markdown per
query. This is bounded and proportional to actual information need.
The upfront cost of designing TypedDict schemas for every type is
unbounded. Tradeoffs favor the docs approach for v1; TypedDict can
come later as a future track if desired.
Track + metadata + state + tracks.md registration for the type-aliases
refactor that follows the audit_weak_types.py findings (430 weak sites
across 29 of 61 files; 86% concentrated in 6 high-traffic files).
Key design decisions (per user approval):
- 10 TypeAlias definitions in src/type_aliases.py (Metadata, CommsLogEntry,
CommsLog, HistoryMessage, History, FileItem, FileItems, ToolDefinition,
ToolCall, CommsLogCallback).
- 1 NamedTuple (FileItemsDiff) for the _reread_file_items return.
- Mechanical replacement of 345 weak sites across 6 files (NOT 430; the
remaining 85 are in 23 lower-impact files deferred to future tracks).
- scripts/audit_weak_types.py gains a --strict mode and a baseline file
(scripts/audit_weak_types.baseline.json) so the count is enforced.
- 2 phases: aliases + 6-file replacement + audit baseline; NamedTuples
+ docs + archive.
- Honest about what's missing: TypedDict / @dataclass migration is a
follow-up track (typed_dict_migration_20260606), not this one.
- Coexistence with the data_oriented_error_handling_20260606 track's
Result[T] / ErrorInfo: the aliases are value-level (data types), Result
is control-level (wrapper). They compose (Result[FileItems] is valid).
No conflict.
Audit baseline:
- Pre-track: 430 weak sites, 0 strong patterns
- Target after Phase 1: ~60 weak sites (only the 23 lower-impact files)
- Top 4 unique type strings account for 86% of findings (4-6 aliases
eliminate the bulk of the noise).
Not blocked by anything; can be executed independently of the other
pending tracks. Blocks typed_dict_migration_20260606 (the future Phase 2).
ed
conductor-tier2