The user pushed back on the v1 recommendation (commit 68354841) twice
in this turn. Both corrections reshape the answer.
Correction 1 (already incorporated): duffle.h + pikuma ps1 are a
C11 STYLE REFERENCE, not an interop pattern. (Captured in v1 §0.)
Correction 2 (NEW, this commit): The C11 path is only worth it under
a hard constraint that no existing Python package can solve. The
shape is request-blob -> C11 pipeline -> response-blob, NOT a
stateful C extension with a Python-facing API. Targets cited:
parsing markdown files/sources into aggregate markdown, context
snapshot processing, "possibly other things."
This commit adds Part 3 (sections 3.1-3.12) to the existing doc.
Part 1 (style) and Part 2 (general interop) stay as background.
Section 4 is re-flagged as "SUPERSEDED - see Part 3".
Part 3 covers:
- The two moves the user's second correction made (threshold-shift
on when, shape-change on what)
- Grounded analysis of the 2 cited targets against actual code:
* src/aggregate.py:380-454 (current markdown hot path is
pure-Python string concat; pyproject.toml has zero
third-party markdown deps)
* src/history.py:1-141 (snapshot processing is bounded
~500KB at 100-snapshot capacity; pickle is the obvious
cheap fix, not C11)
- The request/response wire format design space (text vs binary
vs hybrid envelope-text+payload-binary)
- The pipeline API shape (single C entry point, subprocess-launch
model)
- Revised answer to the "chunkification" question (chunk-array
becomes an internal C implementation detail, not a Python
type)
- Decision tree: profile first, try existing Python packages,
only reach for C11 when hard constraint surfaces
- The 4 questions to revisit when constraint surfaces
- Revised insight: v2 (subprocess + wire format) is strictly
more tractable than v1 (stateful C extension)
- Track implications: chunkification_optimization becomes a
1-page contingency, not a full track; manual_ux_validation
unaffected and confirmed
- v2 verdict matrix (11 rows) replacing v1's 7
Cross-references the actual code paths I read this turn:
- src/aggregate.py:380-454 (build_markdown_from_items)
- src/summarize.py:1-219 (the 3 _summarise_* functions)
- src/history.py:1-141 (UISnapshot, HistoryManager)
- pyproject.toml:6-27 (no markdown deps)
The user is right to push back. The v1 framing was over-engineered.
"Build a stateful C extension" assumed a future need; the actual
answer is "wait for a real bottleneck, then build a simple
subprocess pipeline." The 843-line doc now captures both the
v1 over-engineering AND the v2 contingency plan, so future
sessions can see the iteration and learn from it.
conductor-tier2
12311190b3