# Track Specification: Video Analysis Campaign (2026-06-21) **Status:** Active (spec approved 2026-06-21) **Initialized:** 2026-06-21 **Owner:** Tier 1 Orchestrator (umbrella spec/synthesis); Tier 2 Tech Lead (per-child execution) **Priority:** A (user-blocking; long-running research campaign) **Type:** Multi-track research campaign (1 umbrella + 12 child tracks + 1 synthesis = 14 folders) **Domain:** Meta-tooling (produces research artifacts; no `src/` changes to manual_slop) > **Purpose.** This umbrella track organizes a 12-video research campaign to extract foundational knowledge from the user's preferred sources on AI inference, ML, biological learning, and neuro-compute. The artifacts (transcripts, keyframes, OCR, deep-dive reports) are intermediate inputs to future campaign passes (Pass 2: de-obfuscation; Pass 3: projection to applied domain). **Pass 1 is information extraction + distillation; lossless preservation is the design priority.** > **Companion docs.** This spec is the umbrella. The per-video spec is at `conductor/tracks/video_analysis__20260621/spec.md` (one per child, 12 total). The cross-cutting synthesis spec is at `conductor/tracks/video_analysis_synthesis_20260621/spec.md`. --- ## 0. Campaign Context (multi-pass framing — load-bearing) This is **Pass 1 of 3** in a long-running research campaign. | Pass | Goal | Status | Dependencies | |---|---|---|---| | **1 (THIS)** | Information extraction + distillation. Raw transcripts, keyframes, OCR, deep-dive reports per video. Foundational knowledge base. | Active (this track). | None. | | **2 (FUTURE)** | De-obfuscation via user's custom math encoding notation. Reduce DSL + niche math notation/verbiage into something the user (and associates) can understand. | Not started. **User must rediscover/redefine their encoding system before starting** ("compress/decompress math info" — they have a "handmade" notation from prior work but need to find it). Related: `intent_dsl_survey_20260612`, DSL patterns in `conductor/` docs + track reports. | Blocked by Pass 1. | | **3 (FUTURE)** | Projection to user's applied domain. Apply learnings to user's preferred code style. Influences: handmade / data-oriented / GPGPU community (Timothy Lottes, Onat Türkçüoğlu, Jebrim) + user's own caveats. Some preferences already in `conductor/workflow.md` (data-oriented design styleguide). | Not started. | Blocked by Pass 2. | **Implication for Pass 1 artifacts (load-bearing — read carefully):** - **Raw data MUST be preserved in lossless form.** JSON for transcripts (timestamped), raw images for frames, plain text for OCR. Pass 2 needs every signal. - **Reports should be DETAILED, not summarized.** Per the user directive (2026-06-21), the target is **1000-10000 LOC of markdown per video report**. Over-summarization here is data loss for later. - **Synthesis report preserves detail too.** Pass 2 will compress. - **Don't optimize for "pretty" at the cost of "complete."** --- ## 1. Overview This campaign extracts deep knowledge from 12 YouTube videos the user has curated, organized into 5 thematic clusters: - **Cluster E — Stanford course VODs (>1hr each):** 2 videos - `9vM4p9NN0Ts` — Stanford CS229 — Machine Learning — Building Large Language Models (LLMs) - `lVynu4bo1rY` — Stanford CS336 — Language Modeling from Scratch, Spring 2026, Lecture 3: Architectures - **Cluster A — Math & information-theoretic foundations:** 3 videos - `0yF9TvMeAzM` — Probability Theory is an Extension of Logic - `_U8AwUq_aJQ` — "From Entropy to Epiplexity" (Andrew Wilson and Marc Finzi) - `P75iVMmbqQk` — "Learning Dynamics from Statistics: a score-based approach" (Ludovico Giorgini) - **Cluster B — Platonic / geometric AI representations:** 2 videos - `1mXUFweWOug` — "Towards a Platonic Intelligence with Unified Factored Representations" (Akarsh Kumar) - `K8BmMU1Tm-I` — "Free Lunches: Model Systems for Studying the Agential Gifts from the Platonic Space" (Michael Levin) - **Cluster C — Biological / cognitive / generic systems:** 4 videos - `cDxtFtoQVNc` — The Most Counterintuitive Way to Build a Brain - `YnfaT5APPB0` — "A Multiscale Logic of Collective Intelligence" (Donald Hoffman and Chetan Prakash) - `0BS-BzEFTXA` — "Cognition Emerges from Neural Dynamics" (Earl Miller) - `QeMajYvhEbI` — "Interesting Behavior by Generic Systems" (Chris Fields) - **Cluster D — Applied / practical:** 1 video - `yxkUvXs-hoQ` — Creikey — Deep Learning and Computer Vision for Game Developers (BSC 2025) **Total: 12 videos across 5 clusters.** The campaign delivers: - Per-video: transcript + keyframes + OCR + deep-dive report (1000-10000 LOC markdown each) + summary - Cross-cutting: per-video roll-up + synthesis report with theme matrix, concept map, top takeaways, math prerequisite graph, open questions, and recommended next-watch list --- ## 2. Current State Audit (as of 2026-06-21) ### 2.1 Already Available (DO NOT re-build) | Asset | Location | Status | |---|---|---| | `yt-dlp` (Python module) | NOT installed (system `yt-dlp` binary NOT on PATH either) | **BLOCKER.** Must be installed before any track ships. | | `ffmpeg` 8.1.1 | System PATH | Available. | | `youtube-transcript-api` | Python module | Installed and importable. | | `cv2` (opencv-python) with ffmpeg bindings | `C:\projects\kasa\venv\Lib\site-packages\cv2\` (foreign venv; **DO NOT activate**) | Available — need to install in this repo's venv before any track ships. | | `imagehash`, `PIL` | Foreign venvs only | Need to install in this repo's venv. | | `winsdk` (Windows OCR) | Used by bootslop (`C:\projects\forth\bootslop\process_visuals.py`) | Windows-only; not installed here yet. | | `tesseract` (cross-platform OCR fallback) | Not installed | Optional fallback if `winsdk` proves problematic. | | Reference scripts | `C:\projects\forth\bootslop\download_videos.py`, `extract_frames.py`, `process_visuals.py`, `ocr_interaction.py`, `fetch_blog.py`, `fetch_notes.py` | **Reference only.** New scripts will live in `scripts/video_analysis/` (this repo, per AGENTS.md namespace convention). | | Manual Slop's track convention | `conductor/workflow.md`, `conductor/tracks.md`, nagent/fable/chronology precedents | Established. | | Manual Slop's data-oriented styleguide | `conductor/code_styleguides/data_oriented_design.md` | Referenced by Pass 3 (out of scope here). | | Manual Slop's error-handling convention | `conductor/code_styleguides/error_handling.md` (Result[T] pattern) | Applies to any new Python in `scripts/video_analysis/`. | ### 2.2 Gaps to Fill (this track's scope) | # | Gap | Resolution | |---|---|---| | G1 | No reusable scripts for video download / transcript extraction / keyframe extraction / OCR / report synthesis | Create `scripts/video_analysis/` namespace with 5 scripts | | G2 | No tests for the new scripts | TDD: `tests/test_video_analysis_*.py` (~40-60 tests) | | G3 | No per-video deep-dive reports | 12 child tracks, each producing one `report.md` (1000-10000 LOC) + `summary.md` (200-400 words) | | G4 | No cross-cutting synthesis | 1 synthesis track, blocked by all 12 children, producing `per_video_summary.md` + `report.md` | | G5 | No campaign-level index | `README.md` at umbrella folder with one row per child + status | | G6 | No transcripts/frames/OCR artifacts | Created per-child under `artifacts/` (lossless JSON + raw images) | | G7 | Future-pass hooks not documented | This spec §11 explicitly records the Pass 2/3 dependencies so the next agent can pick up the thread | --- ## 3. Goals 1. **Lossless extraction.** Every signal from the 12 videos (spoken word, on-screen text, keyframes) is captured in a machine-readable form. Pass 2 has all the raw material. 2. **Per-video deep understanding.** Each video gets a 1000-10000 LOC deep-dive report covering: TL;DR, key concepts, frame analysis, transcript highlights, math/theoretical content, cross-video connections, open questions, references. 3. **Cross-cutting synthesis.** A campaign-level report maps themes across the 5 clusters, links concepts between videos, surfaces 5-10 high-level takeaways, and recommends a next-watch list. 4. **Reusable tooling.** The 5 scripts in `scripts/video_analysis/` are independently TDD-tested and usable for any future video analysis (Pass 2, Pass 3, ad-hoc). 5. **No manual_slop `src/` changes.** This is a research campaign; the deliverable is the artifacts and reports. 6. **Future-pass documentation.** This spec records the Pass 2/3 dependencies so the next agent (or the user, after context compaction) has a clear handoff. --- ## 4. Functional Requirements ### FR1. Umbrella folder + README **WHERE:** New folder `conductor/tracks/video_analysis_campaign_20260621/`. **WHAT:** The umbrella folder contains: - `spec.md` (this file) - `plan.md` (campaign-level plan — pointers to children) - `metadata.json` (campaign metadata) - `state.toml` (campaign state) - `README.md` (one row per child + status — like a mini-chronology for the campaign) **The README structure:** ```markdown # Video Analysis Campaign ## Children (in execution order) | # | Slug | Title | Cluster | Track Folder | Status | |---|------|-------|---------|--------------|--------| | 1 | cs229_building_llms | CS229 — Building LLMs | E | [tracks/video_analysis_cs229_building_llms_20260621/](./video_analysis_cs229_building_llms_20260621/) | [~] | | ... | ## Cross-cutting | | Track | Status | |---|-------|--------| | Synthesis | [tracks/video_analysis_synthesis_20260621/](./video_analysis_synthesis_20260621/) | [ ] (blocked by all 12) | ``` ### FR2. 12 child track folders (one per video) **WHERE:** New folders `conductor/tracks/video_analysis__20260621/` (12 total). **WHAT:** Each child folder contains at minimum: - `spec.md` (lightweight — references umbrella, lists the video, specifies what to produce, target LOC) - `artifacts/` (created during execution): - `transcript.json` (timestamped segments + plain text) - `download.log` (yt-dlp log if mp4 downloaded) - `frames/_.jpg` (deduplicated unique frames) - `ocr.md` (full OCR text per frame) - `report.md` (created during execution — 1000-10000 LOC target) - `summary.md` (created during execution — 200-400 words) **Optional (added during execution):** `plan.md`, `metadata.json`, `state.toml` per the standard track convention. **Slug convention:** `` — see `slug_to_url` mapping in §7. ### FR3. 1 synthesis track folder **WHERE:** New folder `conductor/tracks/video_analysis_synthesis_20260621/`. **WHAT:** Contains: - `spec.md` (lightweight — references umbrella, lists the 12 inputs, specifies the synthesis structure) - `per_video_summary.md` (created during execution — one paragraph per video, the "summary of each video" the user requested) - `report.md` (created during execution — the "summary report of key takeaways") **`blocked_by`:** all 12 child tracks (per `state.toml`). ### FR4. Reusable tooling (5 scripts in `scripts/video_analysis/`) Per AGENTS.md: scripts are namespace-isolated by directory. New namespace `scripts/video_analysis/`. | Script | Purpose | Inputs | Outputs | |---|---|---|---| | `scripts/video_analysis/download_video.py` | yt-dlp wrapper (subprocess — no new pyproject deps) | video URL, output path | mp4 file at output path + `download.log` | | `scripts/video_analysis/extract_transcript.py` | youtube-transcript-api wrapper | video URL or ID | `transcript.json` (segments + plain) | | `scripts/video_analysis/extract_keyframes.py` | ffmpeg `select=gt(scene\,0.4)` + cv2 + imagehash dedup | mp4 path, output dir, threshold | `frames/*.jpg` + `extraction_meta.json` | | `scripts/video_analysis/ocr_frames.py` | Windows WinSDK OCR (with tesseract fallback) | frames dir | `ocr.md` (one section per frame) | | `scripts/video_analysis/synthesize_report.py` | Orchestrator — runs the full pipeline for one video | video URL, output dir | `artifacts/` populated + `report.md` stub | **Conventions:** - All scripts follow `conductor/code_styleguides/error_handling.md` (Result[T] pattern — applies to any new Python in `src/` or `scripts/`). - All scripts follow `conductor/code_styleguides/python.md` (1-space indent, type hints, no comments). - All scripts use `subprocess` for yt-dlp / ffmpeg / tesseract (no new pyproject deps). - All scripts support `--help` and a `--json` machine-readable mode for tests. ### FR5. Per-child pipeline (5 phases) Each child track executes: | Phase | Tasks | Output | |---|---|---| | **1. Acquire** | Run `extract_transcript.py` (always succeeds, fast). Run `download_video.py` if frame extraction needs video. | `transcript.json`, `download.log` | | **2. Keyframes** | Run `extract_keyframes.py` with sensible defaults (threshold 0.4). Manual review of frame set. | `frames/*.jpg`, `extraction_meta.json` | | **3. OCR** | Run `ocr_frames.py` on frames. Spot-check OCR quality. | `ocr.md` | | **4. Synthesis** | Tier 3 worker prompt: transcript + OCR + frame images → report.md (target 1000-10000 LOC). Human review + iteration. | `report.md`, `summary.md` | | **5. Verification** | Idempotency check (re-run scripts — should not break). Audit checklist. End-of-track report. | `tests/artifacts//` | ### FR6. Per-video report structure (8 sections, target 1000-10000 LOC) Each `report.md` follows this structure (mirrors `nagent_review`/`fable_review` style): ``` #