The SSDL digest (docs/reports/computational_shapes_ssdl_digest_20260608.md,
504 lines, 30KB) is the theoretical foundation for the chunkification
pattern. Per the digest's Technique 5 "Assume-away (Xar)" in §2.2
and the "Xar-style chunked arrays" recommendation in §5.2, the
chunkification track is a *direct application* of the SSDL's
"assume as much as possible" lens (§4).
This commit adds the SSDL digest to the See Also of the v1+v2
C11-Python interop assessment (front-matter Cross-references line).
The same cross-reference is also being added to:
- conductor/tracks/chunkification_optimization_20260608_PLACEHOLDER/spec.md
(in a new §6.1 "SSDL alignment" subsection)
- conductor/tracks/manual_ux_validation_20260608_PLACEHOLDER/spec.md
(in §5 Architectural Reference + §6 See Also + a new §2.6
"SSDL cross-reference" section that distinguishes GUI ASCII
vocabulary from SSDL vocabulary)
No code modified. Cross-reference only.
Also: small update to conductor/tracks.md to add the 2 new
tracks (manual_ux_validation_20260608_PLACEHOLDER as Active;
chunkification_optimization_20260608_PLACEHOLDER as Backlog/Contingency).
The user pushed back on the v1 recommendation (commit 68354841) twice
in this turn. Both corrections reshape the answer.
Correction 1 (already incorporated): duffle.h + pikuma ps1 are a
C11 STYLE REFERENCE, not an interop pattern. (Captured in v1 §0.)
Correction 2 (NEW, this commit): The C11 path is only worth it under
a hard constraint that no existing Python package can solve. The
shape is request-blob -> C11 pipeline -> response-blob, NOT a
stateful C extension with a Python-facing API. Targets cited:
parsing markdown files/sources into aggregate markdown, context
snapshot processing, "possibly other things."
This commit adds Part 3 (sections 3.1-3.12) to the existing doc.
Part 1 (style) and Part 2 (general interop) stay as background.
Section 4 is re-flagged as "SUPERSEDED - see Part 3".
Part 3 covers:
- The two moves the user's second correction made (threshold-shift
on when, shape-change on what)
- Grounded analysis of the 2 cited targets against actual code:
* src/aggregate.py:380-454 (current markdown hot path is
pure-Python string concat; pyproject.toml has zero
third-party markdown deps)
* src/history.py:1-141 (snapshot processing is bounded
~500KB at 100-snapshot capacity; pickle is the obvious
cheap fix, not C11)
- The request/response wire format design space (text vs binary
vs hybrid envelope-text+payload-binary)
- The pipeline API shape (single C entry point, subprocess-launch
model)
- Revised answer to the "chunkification" question (chunk-array
becomes an internal C implementation detail, not a Python
type)
- Decision tree: profile first, try existing Python packages,
only reach for C11 when hard constraint surfaces
- The 4 questions to revisit when constraint surfaces
- Revised insight: v2 (subprocess + wire format) is strictly
more tractable than v1 (stateful C extension)
- Track implications: chunkification_optimization becomes a
1-page contingency, not a full track; manual_ux_validation
unaffected and confirmed
- v2 verdict matrix (11 rows) replacing v1's 7
Cross-references the actual code paths I read this turn:
- src/aggregate.py:380-454 (build_markdown_from_items)
- src/summarize.py:1-219 (the 3 _summarise_* functions)
- src/history.py:1-141 (UISnapshot, HistoryManager)
- pyproject.toml:6-27 (no markdown deps)
The user is right to push back. The v1 framing was over-engineered.
"Build a stateful C extension" assumed a future need; the actual
answer is "wait for a real bottleneck, then build a simple
subprocess pipeline." The 843-line doc now captures both the
v1 over-engineering AND the v2 contingency plan, so future
sessions can see the iteration and learn from it.
The user asked a sharp, skeptical question: can a chunk-based C11
data structure actually interop with Python's runtime in a way
that's useful for Manual Slop? They explicitly corrected my
first-draft framing (the duffle.h + pikuma ps1 files are a C11
*style reference*, not an interop pattern). The assessment
investigates honestly and reports tractable-vs-not.
docs/reports/c11_python_interop_assessment_20260608.md (564 lines, 38KB):
Part 1: C11 style reference summary
- 11 style observations from reading duffle.h + main.c + pikuma
ps1 duffle/ + hello_gte.c end-to-end
- Byte-width typedef convention (U1/U2/U4/U8, S1/S2/S4/S8, B1-B8, F4/F8)
- The macro meta-DSL (Struct_/Enum_/Array_/Slice_/Opt_/Ret_)
- The I_/IA_/N_ inline discipline
- The r/v pointer rule (restrict OR volatile, never both, never const)
- Slice + Slice_T as the data-structure primitive
- FArena as the allocation primitive (single-buffer, NOT chunked)
- defer/defer_rewind/scope as the cleanup primitive
- KTL (linear key-value table) as the "assume small N" pattern
- What a chunk-array in duffle.h style would look like
Part 2: Interop design space (the actual question)
- 5 candidate interop layers: ctypes, cffi, pybind11, custom
CPython C extension, NumPy wrap
- Honest assessment matrix: build cost, per-op overhead, style
fit, lego-set pattern support
- Verdict: custom CPython C extension is most tractable; pybind11
is style-mismatched; ctypes/cffi work for non-hot-path
- What "MVP chunked C11 package" requires (~500-1000 LOC total)
- 5 questions to ask the user before this becomes a track
- Crucial insight: the user's "unorthodox" interop is most likely
duffle.h-style C11 + thin PyTypeObject glue at the bottom of
the same .h file. Tractable, style-fit high.
Cross-references the 5 sources:
- docs/transcripts/i-h95QIGchY (Reece's Xar reference impl)
- docs/ideation/ed_chunk_data_structures_20260523.md
- docs/reports/session_synthesis_20260608.md (the original proposal)
- src/app_controller.py:716 (the comms.log target)
- The user's local forth_bootslop + pikuma ps1 repos (read in full)
This is a follow-on to the synthesis's 2 proposed tracks
(manual_ux_validation_20260608_PLACEHOLDER + chunkification_optimization_20260608_PLACEHOLDER).
The user's question resolved the "skeptical of #2" concern by
scoping the tractable path: CPython C extension in duffle.h style.
The "lego-set of user-defined Python->C11 chunk ops" is NOT
tractable without a Python->C11 AST emitter, which is a
different (much larger) track.
conductor-tier2