From b184250b78f9d0a050496238ff4174814d19ff46 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Sun, 21 Jun 2026 15:02:44 -0400 Subject: [PATCH] conductor(video_analysis_campaign): Initialize umbrella track + 12 child + 1 synthesis scaffold Pass 1 of 3 user research campaign (12 videos, 5 clusters). - Umbrella: spec.md (full design), plan.md, metadata.json, state.toml, README.md - Multi-pass framing (Pass 2 de-obfuscation, Pass 3 projection) - Lossless preservation directive (1000-10000 LOC per video report target) - Tooling prerequisites: yt-dlp, cv2, imagehash install in repo venv - 5 reusable scripts to live in scripts/video_analysis/ (TDD) - 12 children + 1 synthesis = 14 folders total --- .../README.md | 52 +++ .../metadata.json | 231 +++++++++ .../video_analysis_campaign_20260621/plan.md | 73 +++ .../video_analysis_campaign_20260621/spec.md | 439 ++++++++++++++++++ .../state.toml | 86 ++++ 5 files changed, 881 insertions(+) create mode 100644 conductor/tracks/video_analysis_campaign_20260621/README.md create mode 100644 conductor/tracks/video_analysis_campaign_20260621/metadata.json create mode 100644 conductor/tracks/video_analysis_campaign_20260621/plan.md create mode 100644 conductor/tracks/video_analysis_campaign_20260621/spec.md create mode 100644 conductor/tracks/video_analysis_campaign_20260621/state.toml diff --git a/conductor/tracks/video_analysis_campaign_20260621/README.md b/conductor/tracks/video_analysis_campaign_20260621/README.md new file mode 100644 index 00000000..b52e48c3 --- /dev/null +++ b/conductor/tracks/video_analysis_campaign_20260621/README.md @@ -0,0 +1,52 @@ +# Video Analysis Campaign (2026-06-21) + +**Status:** Active (spec approved 2026-06-21) +**Owner:** Tier 1 Orchestrator (umbrella + synthesis spec); Tier 2 Tech Lead (per-child execution) +**Type:** Multi-track research campaign (14 folders total) + +This is **Pass 1 of 3** in a research campaign to penetrate the AI field. See [spec.md](./spec.md) §0 for the multi-pass framing and §11 for the Pass 2/3 handoff contracts. + +## Children (in execution order) + +| # | Slug | Title | Cluster | Track Folder | Status | +|---|------|-------|---------|--------------|--------| +| 1 | `cs229_building_llms` | Stanford CS229 — Building LLMs | E | [video_analysis_cs229_building_llms_20260621/](./video_analysis_cs229_building_llms_20260621/) | [ ] | +| 2 | `probability_logic` | Probability Theory is an Extension of Logic | A | [video_analysis_probability_logic_20260621/](./video_analysis_probability_logic_20260621/) | [ ] | +| 3 | `entropy_epiplexity` | From Entropy to Epiplexity (Wilson & Finzi) | A | [video_analysis_entropy_epiplexity_20260621/](./video_analysis_entropy_epiplexity_20260621/) | [ ] | +| 4 | `score_dynamics_giorgini` | Learning Dynamics from Statistics (Giorgini) | A | [video_analysis_score_dynamics_giorgini_20260621/](./video_analysis_score_dynamics_giorgini_20260621/) | [ ] | +| 5 | `platonic_intelligence_kumar` | Towards a Platonic Intelligence (Kumar) | B | [video_analysis_platonic_intelligence_kumar_20260621/](./video_analysis_platonic_intelligence_kumar_20260621/) | [ ] | +| 6 | `free_lunches_levin` | Free Lunches (Levin) | B | [video_analysis_free_lunches_levin_20260621/](./video_analysis_free_lunches_levin_20260621/) | [ ] | +| 7 | `generic_systems_fields` | Interesting Behavior by Generic Systems (Fields) | C | [video_analysis_generic_systems_fields_20260621/](./video_analysis_generic_systems_fields_20260621/) | [ ] | +| 8 | `brain_counterintuitive` | Most Counterintuitive Way to Build a Brain | C | [video_analysis_brain_counterintuitive_20260621/](./video_analysis_brain_counterintuitive_20260621/) | [ ] | +| 9 | `neural_dynamics_miller` | Cognition Emerges from Neural Dynamics (Miller) | C | [video_analysis_neural_dynamics_miller_20260621/](./video_analysis_neural_dynamics_miller_20260621/) | [ ] | +| 10 | `multiscale_hoffman` | Multiscale Logic of Collective Intelligence (Hoffman & Prakash) | C | [video_analysis_multiscale_hoffman_20260621/](./video_analysis_multiscale_hoffman_20260621/) | [ ] | +| 11 | `cs336_architectures` | Stanford CS336 Lecture 3: Architectures | E | [video_analysis_cs336_architectures_20260621/](./video_analysis_cs336_architectures_20260621/) | [ ] | +| 12 | `creikey_dl_cv` | Creikey — DL/CV for Game Developers | D | [video_analysis_creikey_dl_cv_20260621/](./video_analysis_creikey_dl_cv_20260621/) | [ ] | + +## Cross-cutting + +| | Track | Status | +|---|-------|--------| +| Synthesis (blocked by all 12) | [video_analysis_synthesis_20260621/](./video_analysis_synthesis_20260621/) | [ ] | + +## Status legend + +- `[ ]` — not started +- `[~]` — in progress +- `[x]` — shipped +- `[!]` — blocked + +## Cluster legend + +- **A** — Math & information-theoretic foundations (3 videos) +- **B** — Platonic / geometric AI representations (2 videos) +- **C** — Biological / cognitive / generic systems (4 videos) +- **D** — Applied / practical (1 video) +- **E** — Stanford course VODs >1hr (2 videos) + +## See also + +- [spec.md](./spec.md) — full design (Overview, Current State Audit, Goals, FRs, NFRs, Architecture, Future-Pass Hooks, Risk Register, User Directives) +- [plan.md](./plan.md) — campaign-level plan (Phases 0-4) +- [metadata.json](./metadata.json) — scope, verification criteria, risk register +- [state.toml](./state.toml) — current phase + task tracking diff --git a/conductor/tracks/video_analysis_campaign_20260621/metadata.json b/conductor/tracks/video_analysis_campaign_20260621/metadata.json new file mode 100644 index 00000000..e9d92a3b --- /dev/null +++ b/conductor/tracks/video_analysis_campaign_20260621/metadata.json @@ -0,0 +1,231 @@ +{ + "track_id": "video_analysis_campaign_20260621", + "name": "Video Analysis Campaign (12 videos, 5 clusters, 3 passes)", + "created": "2026-06-21", + "status": "spec_approved", + "blocked_by": [], + "blocks": [ + "video_analysis_synthesis_20260621" + ], + "priority": "A", + "rationale": "User-blocking research campaign to extract foundational knowledge from 12 curated YouTube videos on AI inference, ML, biological learning, and neuro-compute. The artifacts feed future Pass 2 (de-obfuscation via user's math encoding) and Pass 3 (projection to applied domain). Lossless preservation is the design priority. Research-only — no src/ changes, no new pyproject deps (all tools via subprocess or existing venv).", + "type": "multi-track research campaign (1 umbrella + 12 child tracks + 1 synthesis = 14 folders total)", + "domain": "meta-tooling (research artifacts; no manual_slop src/ changes)", + "scope": { + "new_folders": [ + "conductor/tracks/video_analysis_campaign_20260621/", + "conductor/tracks/video_analysis_cs229_building_llms_20260621/", + "conductor/tracks/video_analysis_probability_logic_20260621/", + "conductor/tracks/video_analysis_entropy_epiplexity_20260621/", + "conductor/tracks/video_analysis_score_dynamics_giorgini_20260621/", + "conductor/tracks/video_analysis_platonic_intelligence_kumar_20260621/", + "conductor/tracks/video_analysis_free_lunches_levin_20260621/", + "conductor/tracks/video_analysis_generic_systems_fields_20260621/", + "conductor/tracks/video_analysis_brain_counterintuitive_20260621/", + "conductor/tracks/video_analysis_neural_dynamics_miller_20260621/", + "conductor/tracks/video_analysis_multiscale_hoffman_20260621/", + "conductor/tracks/video_analysis_cs336_architectures_20260621/", + "conductor/tracks/video_analysis_creikey_dl_cv_20260621/", + "conductor/tracks/video_analysis_synthesis_20260621/" + ], + "new_files_per_child": [ + "spec.md (lightweight)", + "artifacts/transcript.json", + "artifacts/ocr.md", + "artifacts/frames/_.jpg (deduplicated)", + "artifacts/extraction_meta.json", + "report.md (1000-10000 LOC target)", + "summary.md (200-400 words)" + ], + "new_files_scripts": [ + "scripts/video_analysis/download_video.py", + "scripts/video_analysis/extract_transcript.py", + "scripts/video_analysis/extract_keyframes.py", + "scripts/video_analysis/ocr_frames.py", + "scripts/video_analysis/synthesize_report.py" + ], + "new_files_tests": [ + "tests/test_video_analysis_download_video.py", + "tests/test_video_analysis_extract_transcript.py", + "tests/test_video_analysis_extract_keyframes.py", + "tests/test_video_analysis_ocr_frames.py", + "tests/test_video_analysis_synthesize_report.py" + ], + "new_files_synthesis": [ + "conductor/tracks/video_analysis_synthesis_20260621/spec.md", + "conductor/tracks/video_analysis_synthesis_20260621/per_video_summary.md", + "conductor/tracks/video_analysis_synthesis_20260621/report.md" + ], + "modified_files": [], + "deleted_files": [], + "gitignored_patterns": [ + "*.mp4 (video files - too large for git)", + "artifacts/frames/*.jpg if >500KB each", + "tests/artifacts// (per AGENTS.md artifact isolation)" + ] + }, + "estimated_effort": { + "method": "scope (per conductor/workflow.md Tier 1 Track Initialization Rules). NO day estimates.", + "phase_0": "4 tasks: tooling prerequisites (yt-dlp, cv2, imagehash, OCR backend decision)", + "phase_1": "10 tasks: 5 reusable scripts with TDD (red + green per script)", + "phase_2": "12 child tracks × 5 phases each = 60 child track execution tasks (tracked in child plans, not this umbrella)", + "phase_3": "1 synthesis track (blocked by all 12 children)", + "phase_4": "4 tasks: campaign closeout (README update, end-of-track report, archive move, chronology update)", + "summary": "14 track folders (1 umbrella + 12 children + 1 synthesis), 5 reusable scripts, ~40-60 unit tests, 12 reports (1000-10000 LOC each), 12 summaries, 1 cross-cutting synthesis report. No day estimates per project convention." + }, + "verification_criteria": [ + "yt-dlp installed and importable in this repo's venv", + "cv2, imagehash, PIL installed in this repo's venv", + "OCR backend chosen (winsdk or tesseract) and working", + "All 5 scripts in scripts/video_analysis/ have passing TDD tests", + "All 12 child tracks shipped: each has transcript.json, frames/, ocr.md, report.md (1000-10000 LOC), summary.md", + "Synthesis track shipped: per_video_summary.md + report.md", + "Umbrella README.md shows all 12 children + synthesis as shipped", + "End-of-track report at docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md", + "All artifacts preserved losslessly (JSON for transcripts, raw images for frames, plain text for OCR)", + "No src/*.py files created or modified (per AGENTS.md File Size and Naming Convention)", + "No new pyproject.toml dependencies (all tools via subprocess or existing venv)", + "Future-pass hooks (§11 of spec.md) intact and documented for Pass 2/3" + ], + "risk_register": [ + { + "id": "R1", + "title": "yt-dlp not installed locally", + "likelihood": "high", + "scope_impact": "First child track blocked until installed", + "mitigation": "Install via pip install yt-dlp in this repo's venv (single one-time task in Phase 0)" + }, + { + "id": "R2", + "title": "OCR quality insufficient for technical content", + "likelihood": "medium", + "scope_impact": "Some frames may have illegible text", + "mitigation": "Spot-check OCR per frame; manually transcribe critical frames in the report.md section" + }, + { + "id": "R3", + "title": "Report exceeds 10000 LOC target", + "likelihood": "low", + "scope_impact": "User may want to split", + "mitigation": "Pass 2 can split; Pass 1 should not artificially cap" + }, + { + "id": "R4", + "title": "Video mp4 files exceed disk space", + "likelihood": "medium", + "scope_impact": "Could hit quota", + "mitigation": "Delete mp4 after frame extraction (extract_keyframes.py should do this)" + }, + { + "id": "R5", + "title": "Two videos failed oEmbed fetch", + "likelihood": "confirmed for 9vM4p9NN0Ts and lVynu4bo1rY", + "scope_impact": "Unknown until track execution", + "mitigation": "User confirmed identities. yt-dlp may still work (different from oEmbed). Verify in Phase 1 of each track." + }, + { + "id": "R6", + "title": "User's math encoding notation (Pass 2) lost", + "likelihood": "medium", + "scope_impact": "Blocks Pass 2", + "mitigation": "User action item: rediscover/redefine encoding before Pass 2 starts. Recorded in spec.md §11.1." + }, + { + "id": "R7", + "title": "Pass 1 over-summarization loses signal for Pass 2", + "likelihood": "medium (if not enforced)", + "scope_impact": "Cascades to Pass 2/3", + "mitigation": "The 1000-10000 LOC target + spec.md §0 explicit warning + per-section completeness check in verification" + }, + { + "id": "R8", + "title": "Tier 2 capacity for 12+ child tracks", + "likelihood": "medium", + "scope_impact": "Tracks ship in sequence", + "mitigation": "Each child is independently shippable; the campaign is async" + }, + { + "id": "R9", + "title": "Transcript API rate-limiting", + "likelihood": "low", + "scope_impact": "Some videos may fail on first fetch", + "mitigation": "Retry with backoff in extract_transcript.py" + }, + { + "id": "R10", + "title": "cv2 / imagehash not in this repo's venv", + "likelihood": "high (verified - exist only in foreign venvs)", + "scope_impact": "Blocks keyframe extraction", + "mitigation": "Install via pip install opencv-python imagehash pillow in this repo's venv (single one-time task in Phase 0)" + } + ], + "architecture_reference": { + "primary_documents": [ + "conductor/workflow.md (track convention, per-task commits, git notes, verification protocol)", + "conductor/code_styleguides/python.md (1-space indent, type hints, no comments)", + "conductor/code_styleguides/error_handling.md (Result[T] pattern for new scripts)", + "AGENTS.md (artifact isolation, file naming, no new src/.py)", + "conductor/chronology.md (after campaign ships, 14 new rows added here)" + ], + "related_tracks": [ + "conductor/tracks/intent_dsl_survey_20260612/ (Pass 2 may build on this)", + "conductor/tracks/nagent_review_20260608/ (precedent for deep-dive report format)", + "conductor/tracks/fable_review_20260617/ (precedent for synthesis report format)", + "conductor/tracks/chronology_20260619/ (precedent for spec/plan/metadata/state schema)" + ], + "external_references": [ + "C:/projects/forth/bootslop/download_videos.py (yt-dlp usage reference)", + "C:/projects/forth/bootslop/extract_frames.py (imagehash + cv2 keyframe extraction reference)", + "C:/projects/forth/bootslop/process_visuals.py (winsdk OCR + visual analysis reference)", + "C:/projects/forth/bootslop/ocr_interaction.py (standalone OCR reference)", + "https://pypi.org/project/youtube-transcript-api/", + "C:/projects/kasa/venv/Lib/site-packages/cv2/opencv_videoio_ffmpeg481_64.dll (proves cv2/ffmpeg installs on this machine)" + ], + "styleguides_applied": [ + "data_oriented_design.md (referenced by Pass 3, not directly by Pass 1)", + "python.md (1-space indent for all new scripts)", + "error_handling.md (Result[T] for all new scripts)", + "feature_flags.md (scripts are file-presence, no config flags needed)", + "workspace_paths.md (test artifacts in tests/artifacts/)" + ] + }, + "deferred_to_followup_tracks": [ + { + "title": "Pass 2: De-obfuscation via user's math encoding notation", + "description": "Apply the user's custom math encoding/compression notation to reduce DSL + niche math notation/verbiage into something the user can understand. Consumes all Pass 1 artifacts.", + "track_status": "not started - blocked by this track", + "blocker_action_item": "User must rediscover/redefine their 'compress/decompress math info' encoding notation before Pass 2 starts. See spec.md §11.1." + }, + { + "title": "Pass 3: Projection to user's applied domain", + "description": "Apply Pass 2 outputs to user's preferred code style. Influences: handmade/data-oriented/GPGPU community (Lottes, Onat, Jebrim) + user's own caveats.", + "track_status": "not started - blocked by Pass 2", + "blocker_action_item": "User must articulate 'own caveats' before Pass 3 starts. See spec.md §11.2." + } + ], + "regressions_and_pre_existing_failures": [], + "pre_existing_failures_remaining": [], + "user_directives": [ + "Order confirmed (12-video sequence, 2026-06-21)", + "Report target: minimum 1000 LOC, maximum 10000 LOC markdown per video (2026-06-21)", + "Multi-pass framing: Pass 1 = information extraction (this track), Pass 2 = de-obfuscation, Pass 3 = projection (2026-06-21)", + "Pass 1 artifacts must be lossless - over-summarization is data loss for Pass 2 (2026-06-21)", + "Stanford CS229 = 9vM4p9NN0Ts, Stanford CS336 Lecture 3 = lVynu4bo1rY (user-confirmed mapping, 2026-06-21)", + "Future-pass hooks must be explicit in spec.md so the next agent / future-self can pick up the thread (2026-06-21)", + "No day estimates per conductor/workflow.md Tier 1 Track Initialization Rules (added 2026-06-16). Scope measured in files/sites only." + ], + "videos": [ + {"order": 1, "slug": "cs229_building_llms", "cluster": "E", "title": "Stanford CS229 - Building Large Language Models (LLMs)", "youtube_id": "9vM4p9NN0Ts", "author": "Stanford CS229"}, + {"order": 2, "slug": "probability_logic", "cluster": "A", "title": "Probability Theory is an Extension of Logic", "youtube_id": "0yF9TvMeAzM", "author": null}, + {"order": 3, "slug": "entropy_epiplexity", "cluster": "A", "title": "From Entropy to Epiplexity", "youtube_id": "_U8AwUq_aJQ", "author": "Andrew Wilson and Marc Finzi"}, + {"order": 4, "slug": "score_dynamics_giorgini", "cluster": "A", "title": "Learning Dynamics from Statistics: a score-based approach", "youtube_id": "P75iVMmbqQk", "author": "Ludovico Giorgini"}, + {"order": 5, "slug": "platonic_intelligence_kumar", "cluster": "B", "title": "Towards a Platonic Intelligence with Unified Factored Representations", "youtube_id": "1mXUFweWOug", "author": "Akarsh Kumar"}, + {"order": 6, "slug": "free_lunches_levin", "cluster": "B", "title": "Free Lunches: Model Systems for Studying the Agential Gifts from the Platonic Space", "youtube_id": "K8BmMU1Tm-I", "author": "Michael Levin"}, + {"order": 7, "slug": "generic_systems_fields", "cluster": "C", "title": "Interesting Behavior by Generic Systems", "youtube_id": "QeMajYvhEbI", "author": "Chris Fields"}, + {"order": 8, "slug": "brain_counterintuitive", "cluster": "C", "title": "The Most Counterintuitive Way to Build a Brain", "youtube_id": "cDxtFtoQVNc", "author": null}, + {"order": 9, "slug": "neural_dynamics_miller", "cluster": "C", "title": "Cognition Emerges from Neural Dynamics", "youtube_id": "0BS-BzEFTXA", "author": "Earl Miller"}, + {"order": 10, "slug": "multiscale_hoffman", "cluster": "C", "title": "A Multiscale Logic of Collective Intelligence", "youtube_id": "YnfaT5APPB0", "author": "Donald Hoffman and Chetan Prakash"}, + {"order": 11, "slug": "cs336_architectures", "cluster": "E", "title": "Stanford CS336 Lecture 3: Architectures", "youtube_id": "lVynu4bo1rY", "author": "Stanford CS336 Spring 2026"}, + {"order": 12, "slug": "creikey_dl_cv", "cluster": "D", "title": "Creikey - Deep Learning and Computer Vision for Game Developers (BSC 2025)", "youtube_id": "yxkUvXs-hoQ", "author": "Creikey"} + ] +} diff --git a/conductor/tracks/video_analysis_campaign_20260621/plan.md b/conductor/tracks/video_analysis_campaign_20260621/plan.md new file mode 100644 index 00000000..116aecf5 --- /dev/null +++ b/conductor/tracks/video_analysis_campaign_20260621/plan.md @@ -0,0 +1,73 @@ +# Plan: Video Analysis Campaign (umbrella) + +This is the umbrella-level plan. Per the Tier 1 Track Initialization Rules, scope is measured in files/sites — no day estimates. + +## Phase 0: Tooling Prerequisites + +Must be completed before any child track can ship. One-time setup. + +- [ ] **Task 0.1:** Install `yt-dlp` in this repo's venv (`pip install yt-dlp`). Verify with `python -c "import yt_dlp; print(yt_dlp.version.__version__)"`. +- [ ] **Task 0.2:** Install `opencv-python`, `imagehash`, `pillow` in this repo's venv. Verify imports. +- [ ] **Task 0.3:** Decide on OCR backend. Try `winsdk` first (matches bootslop); fall back to `tesseract` if `winsdk` proves problematic. +- [ ] **Task 0.4:** Create `scripts/video_analysis/` namespace and `tests/test_video_analysis_*.py` skeleton. + +## Phase 1: Reusable Tooling (5 scripts, TDD) + +Each script is independently TDD-tested. Order: extract_transcript → download_video → extract_keyframes → ocr_frames → synthesize_report (synthesize_report is last because it composes the others). + +- [ ] **Task 1.1:** Write tests for `extract_transcript.py` (red). Tests cover: success path, network error, missing video ID, malformed JSON response, retry behavior. +- [ ] **Task 1.2:** Implement `extract_transcript.py` (green). CLI: `--url`, `--output`, `--json`. Outputs `transcript.json` with `segments` (list of `{start, duration, text}`) + `plain` (joined text) + `metadata` (video ID, fetch timestamp). +- [ ] **Task 1.3:** Write tests for `download_video.py` (red). +- [ ] **Task 1.4:** Implement `download_video.py` (green). CLI: `--url`, `--output`, `--json`. Subprocess `yt-dlp`. Outputs `download.log`. +- [ ] **Task 1.5:** Write tests for `extract_keyframes.py` (red). +- [ ] **Task 1.6:** Implement `extract_keyframes.py` (green). CLI: `--video`, `--output-dir`, `--threshold`, `--json`. Uses ffmpeg `select=gt(scene\,0.4)` + cv2 frame extraction + imagehash dedup. Outputs `frames/*.jpg` + `extraction_meta.json`. +- [ ] **Task 1.7:** Write tests for `ocr_frames.py` (red). +- [ ] **Task 1.8:** Implement `ocr_frames.py` (green). CLI: `--frames-dir`, `--output`, `--json`. Uses winsdk (with tesseract fallback). Outputs `ocr.md` with one section per frame (image path + OCR text + timestamp). +- [ ] **Task 1.9:** Write tests for `synthesize_report.py` (red). +- [ ] **Task 1.10:** Implement `synthesize_report.py` (green). CLI: `--url`, `--slug`, `--output-dir`, `--json`. Orchestrates the full pipeline for one video. Outputs `artifacts/` populated + `report.md` stub + `summary.md` stub. + +## Phase 2: Per-Child Tracks (12 tracks) + +Each child track follows the 5-phase pipeline. The Tier 2 Tech Lead executes these. The umbrella plan does NOT enumerate per-child tasks — those live in each child's `plan.md` (created during execution). + +**Execution order** (per §6 of spec.md): + +| # | Slug | Cluster | Notes | +|---|------|---------|-------| +| 1 | `cs229_building_llms` | E | Verify yt-dlp access (oEmbed failed 401) | +| 2 | `probability_logic` | A | | +| 3 | `entropy_epiplexity` | A | | +| 4 | `score_dynamics_giorgini` | A | | +| 5 | `platonic_intelligence_kumar` | B | | +| 6 | `free_lunches_levin` | B | | +| 7 | `generic_systems_fields` | C | | +| 8 | `brain_counterintuitive` | C | | +| 9 | `neural_dynamics_miller` | C | | +| 10 | `multiscale_hoffman` | C | | +| 11 | `cs336_architectures` | E | Verify yt-dlp access (oEmbed failed 401) | +| 12 | `creikey_dl_cv` | D | | + +**Note on E-cluster yt-dlp verification:** the oEmbed API returned 401 for `9vM4p9NN0Ts` and `lVynu4bo1rY`. This may be an oEmbed-specific restriction (private/age-restricted); `yt-dlp` may still work. Phase 1 of those child tracks must verify yt-dlp access before downloading anything. + +## Phase 3: Synthesis Track (blocked by all 12 children) + +After all 12 child tracks ship, the synthesis track consumes their outputs and produces: +- `per_video_summary.md` (the "summary of each video" the user requested) +- `report.md` (the "summary report of key takeaways" — theme matrix, concept map, top takeaways, math prereq graph, open questions, next-watch list) + +## Phase 4: Campaign Closeout + +- [ ] **Task 4.1:** Update umbrella `README.md` with final statuses (all 12 children + synthesis shipped). +- [ ] **Task 4.2:** Write end-of-track report at `docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md`. +- [ ] **Task 4.3:** Move umbrella + 13 children to `conductor/archive/` per the project's archiving convention. +- [ ] **Task 4.4:** Update `conductor/chronology.md` with the 14 track rows. + +## Verification (gate per workflow.md) + +Each child track's Phase 5 must include: +- [ ] Idempotency check: re-running scripts produces identical outputs (modulo timestamps) +- [ ] Audit checklist: every section of the report is populated, no "TBD" +- [ ] Tests pass +- [ ] Per-task commits with git notes + +The campaign-level verification (Phase 4.2 end-of-track report) confirms all 12 children + 1 synthesis are shipped and the future-pass hooks (§11 of spec.md) are intact. diff --git a/conductor/tracks/video_analysis_campaign_20260621/spec.md b/conductor/tracks/video_analysis_campaign_20260621/spec.md new file mode 100644 index 00000000..2a1b61c6 --- /dev/null +++ b/conductor/tracks/video_analysis_campaign_20260621/spec.md @@ -0,0 +1,439 @@ +# Track Specification: Video Analysis Campaign (2026-06-21) + +**Status:** Active (spec approved 2026-06-21) +**Initialized:** 2026-06-21 +**Owner:** Tier 1 Orchestrator (umbrella spec/synthesis); Tier 2 Tech Lead (per-child execution) +**Priority:** A (user-blocking; long-running research campaign) +**Type:** Multi-track research campaign (1 umbrella + 12 child tracks + 1 synthesis = 14 folders) +**Domain:** Meta-tooling (produces research artifacts; no `src/` changes to manual_slop) + +> **Purpose.** This umbrella track organizes a 12-video research campaign to extract foundational knowledge from the user's preferred sources on AI inference, ML, biological learning, and neuro-compute. The artifacts (transcripts, keyframes, OCR, deep-dive reports) are intermediate inputs to future campaign passes (Pass 2: de-obfuscation; Pass 3: projection to applied domain). **Pass 1 is information extraction + distillation; lossless preservation is the design priority.** + +> **Companion docs.** This spec is the umbrella. The per-video spec is at `conductor/tracks/video_analysis__20260621/spec.md` (one per child, 12 total). The cross-cutting synthesis spec is at `conductor/tracks/video_analysis_synthesis_20260621/spec.md`. + +--- + +## 0. Campaign Context (multi-pass framing — load-bearing) + +This is **Pass 1 of 3** in a long-running research campaign. + +| Pass | Goal | Status | Dependencies | +|---|---|---|---| +| **1 (THIS)** | Information extraction + distillation. Raw transcripts, keyframes, OCR, deep-dive reports per video. Foundational knowledge base. | Active (this track). | None. | +| **2 (FUTURE)** | De-obfuscation via user's custom math encoding notation. Reduce DSL + niche math notation/verbiage into something the user (and associates) can understand. | Not started. **User must rediscover/redefine their encoding system before starting** ("compress/decompress math info" — they have a "handmade" notation from prior work but need to find it). Related: `intent_dsl_survey_20260612`, DSL patterns in `conductor/` docs + track reports. | Blocked by Pass 1. | +| **3 (FUTURE)** | Projection to user's applied domain. Apply learnings to user's preferred code style. Influences: handmade / data-oriented / GPGPU community (Timothy Lottes, Onat Türkçüoğlu, Jebrim) + user's own caveats. Some preferences already in `conductor/workflow.md` (data-oriented design styleguide). | Not started. | Blocked by Pass 2. | + +**Implication for Pass 1 artifacts (load-bearing — read carefully):** +- **Raw data MUST be preserved in lossless form.** JSON for transcripts (timestamped), raw images for frames, plain text for OCR. Pass 2 needs every signal. +- **Reports should be DETAILED, not summarized.** Per the user directive (2026-06-21), the target is **1000-10000 LOC of markdown per video report**. Over-summarization here is data loss for later. +- **Synthesis report preserves detail too.** Pass 2 will compress. +- **Don't optimize for "pretty" at the cost of "complete."** + +--- + +## 1. Overview + +This campaign extracts deep knowledge from 12 YouTube videos the user has curated, organized into 5 thematic clusters: + +- **Cluster E — Stanford course VODs (>1hr each):** 2 videos + - `9vM4p9NN0Ts` — Stanford CS229 — Machine Learning — Building Large Language Models (LLMs) + - `lVynu4bo1rY` — Stanford CS336 — Language Modeling from Scratch, Spring 2026, Lecture 3: Architectures +- **Cluster A — Math & information-theoretic foundations:** 3 videos + - `0yF9TvMeAzM` — Probability Theory is an Extension of Logic + - `_U8AwUq_aJQ` — "From Entropy to Epiplexity" (Andrew Wilson and Marc Finzi) + - `P75iVMmbqQk` — "Learning Dynamics from Statistics: a score-based approach" (Ludovico Giorgini) +- **Cluster B — Platonic / geometric AI representations:** 2 videos + - `1mXUFweWOug` — "Towards a Platonic Intelligence with Unified Factored Representations" (Akarsh Kumar) + - `K8BmMU1Tm-I` — "Free Lunches: Model Systems for Studying the Agential Gifts from the Platonic Space" (Michael Levin) +- **Cluster C — Biological / cognitive / generic systems:** 4 videos + - `cDxtFtoQVNc` — The Most Counterintuitive Way to Build a Brain + - `YnfaT5APPB0` — "A Multiscale Logic of Collective Intelligence" (Donald Hoffman and Chetan Prakash) + - `0BS-BzEFTXA` — "Cognition Emerges from Neural Dynamics" (Earl Miller) + - `QeMajYvhEbI` — "Interesting Behavior by Generic Systems" (Chris Fields) +- **Cluster D — Applied / practical:** 1 video + - `yxkUvXs-hoQ` — Creikey — Deep Learning and Computer Vision for Game Developers (BSC 2025) + +**Total: 12 videos across 5 clusters.** + +The campaign delivers: +- Per-video: transcript + keyframes + OCR + deep-dive report (1000-10000 LOC markdown each) + summary +- Cross-cutting: per-video roll-up + synthesis report with theme matrix, concept map, top takeaways, math prerequisite graph, open questions, and recommended next-watch list + +--- + +## 2. Current State Audit (as of 2026-06-21) + +### 2.1 Already Available (DO NOT re-build) + +| Asset | Location | Status | +|---|---|---| +| `yt-dlp` (Python module) | NOT installed (system `yt-dlp` binary NOT on PATH either) | **BLOCKER.** Must be installed before any track ships. | +| `ffmpeg` 8.1.1 | System PATH | Available. | +| `youtube-transcript-api` | Python module | Installed and importable. | +| `cv2` (opencv-python) with ffmpeg bindings | `C:\projects\kasa\venv\Lib\site-packages\cv2\` (foreign venv; **DO NOT activate**) | Available — need to install in this repo's venv before any track ships. | +| `imagehash`, `PIL` | Foreign venvs only | Need to install in this repo's venv. | +| `winsdk` (Windows OCR) | Used by bootslop (`C:\projects\forth\bootslop\process_visuals.py`) | Windows-only; not installed here yet. | +| `tesseract` (cross-platform OCR fallback) | Not installed | Optional fallback if `winsdk` proves problematic. | +| Reference scripts | `C:\projects\forth\bootslop\download_videos.py`, `extract_frames.py`, `process_visuals.py`, `ocr_interaction.py`, `fetch_blog.py`, `fetch_notes.py` | **Reference only.** New scripts will live in `scripts/video_analysis/` (this repo, per AGENTS.md namespace convention). | +| Manual Slop's track convention | `conductor/workflow.md`, `conductor/tracks.md`, nagent/fable/chronology precedents | Established. | +| Manual Slop's data-oriented styleguide | `conductor/code_styleguides/data_oriented_design.md` | Referenced by Pass 3 (out of scope here). | +| Manual Slop's error-handling convention | `conductor/code_styleguides/error_handling.md` (Result[T] pattern) | Applies to any new Python in `scripts/video_analysis/`. | + +### 2.2 Gaps to Fill (this track's scope) + +| # | Gap | Resolution | +|---|---|---| +| G1 | No reusable scripts for video download / transcript extraction / keyframe extraction / OCR / report synthesis | Create `scripts/video_analysis/` namespace with 5 scripts | +| G2 | No tests for the new scripts | TDD: `tests/test_video_analysis_*.py` (~40-60 tests) | +| G3 | No per-video deep-dive reports | 12 child tracks, each producing one `report.md` (1000-10000 LOC) + `summary.md` (200-400 words) | +| G4 | No cross-cutting synthesis | 1 synthesis track, blocked by all 12 children, producing `per_video_summary.md` + `report.md` | +| G5 | No campaign-level index | `README.md` at umbrella folder with one row per child + status | +| G6 | No transcripts/frames/OCR artifacts | Created per-child under `artifacts/` (lossless JSON + raw images) | +| G7 | Future-pass hooks not documented | This spec §11 explicitly records the Pass 2/3 dependencies so the next agent can pick up the thread | + +--- + +## 3. Goals + +1. **Lossless extraction.** Every signal from the 12 videos (spoken word, on-screen text, keyframes) is captured in a machine-readable form. Pass 2 has all the raw material. +2. **Per-video deep understanding.** Each video gets a 1000-10000 LOC deep-dive report covering: TL;DR, key concepts, frame analysis, transcript highlights, math/theoretical content, cross-video connections, open questions, references. +3. **Cross-cutting synthesis.** A campaign-level report maps themes across the 5 clusters, links concepts between videos, surfaces 5-10 high-level takeaways, and recommends a next-watch list. +4. **Reusable tooling.** The 5 scripts in `scripts/video_analysis/` are independently TDD-tested and usable for any future video analysis (Pass 2, Pass 3, ad-hoc). +5. **No manual_slop `src/` changes.** This is a research campaign; the deliverable is the artifacts and reports. +6. **Future-pass documentation.** This spec records the Pass 2/3 dependencies so the next agent (or the user, after context compaction) has a clear handoff. + +--- + +## 4. Functional Requirements + +### FR1. Umbrella folder + README + +**WHERE:** New folder `conductor/tracks/video_analysis_campaign_20260621/`. + +**WHAT:** The umbrella folder contains: +- `spec.md` (this file) +- `plan.md` (campaign-level plan — pointers to children) +- `metadata.json` (campaign metadata) +- `state.toml` (campaign state) +- `README.md` (one row per child + status — like a mini-chronology for the campaign) + +**The README structure:** +```markdown +# Video Analysis Campaign + +## Children (in execution order) + +| # | Slug | Title | Cluster | Track Folder | Status | +|---|------|-------|---------|--------------|--------| +| 1 | cs229_building_llms | CS229 — Building LLMs | E | [tracks/video_analysis_cs229_building_llms_20260621/](./video_analysis_cs229_building_llms_20260621/) | [~] | +| ... | + +## Cross-cutting + +| | Track | Status | +|---|-------|--------| +| Synthesis | [tracks/video_analysis_synthesis_20260621/](./video_analysis_synthesis_20260621/) | [ ] (blocked by all 12) | +``` + +### FR2. 12 child track folders (one per video) + +**WHERE:** New folders `conductor/tracks/video_analysis__20260621/` (12 total). + +**WHAT:** Each child folder contains at minimum: +- `spec.md` (lightweight — references umbrella, lists the video, specifies what to produce, target LOC) +- `artifacts/` (created during execution): + - `transcript.json` (timestamped segments + plain text) + - `download.log` (yt-dlp log if mp4 downloaded) + - `frames/_.jpg` (deduplicated unique frames) + - `ocr.md` (full OCR text per frame) +- `report.md` (created during execution — 1000-10000 LOC target) +- `summary.md` (created during execution — 200-400 words) + +**Optional (added during execution):** `plan.md`, `metadata.json`, `state.toml` per the standard track convention. + +**Slug convention:** `` — see `slug_to_url` mapping in §7. + +### FR3. 1 synthesis track folder + +**WHERE:** New folder `conductor/tracks/video_analysis_synthesis_20260621/`. + +**WHAT:** Contains: +- `spec.md` (lightweight — references umbrella, lists the 12 inputs, specifies the synthesis structure) +- `per_video_summary.md` (created during execution — one paragraph per video, the "summary of each video" the user requested) +- `report.md` (created during execution — the "summary report of key takeaways") + +**`blocked_by`:** all 12 child tracks (per `state.toml`). + +### FR4. Reusable tooling (5 scripts in `scripts/video_analysis/`) + +Per AGENTS.md: scripts are namespace-isolated by directory. New namespace `scripts/video_analysis/`. + +| Script | Purpose | Inputs | Outputs | +|---|---|---|---| +| `scripts/video_analysis/download_video.py` | yt-dlp wrapper (subprocess — no new pyproject deps) | video URL, output path | mp4 file at output path + `download.log` | +| `scripts/video_analysis/extract_transcript.py` | youtube-transcript-api wrapper | video URL or ID | `transcript.json` (segments + plain) | +| `scripts/video_analysis/extract_keyframes.py` | ffmpeg `select=gt(scene\,0.4)` + cv2 + imagehash dedup | mp4 path, output dir, threshold | `frames/*.jpg` + `extraction_meta.json` | +| `scripts/video_analysis/ocr_frames.py` | Windows WinSDK OCR (with tesseract fallback) | frames dir | `ocr.md` (one section per frame) | +| `scripts/video_analysis/synthesize_report.py` | Orchestrator — runs the full pipeline for one video | video URL, output dir | `artifacts/` populated + `report.md` stub | + +**Conventions:** +- All scripts follow `conductor/code_styleguides/error_handling.md` (Result[T] pattern — applies to any new Python in `src/` or `scripts/`). +- All scripts follow `conductor/code_styleguides/python.md` (1-space indent, type hints, no comments). +- All scripts use `subprocess` for yt-dlp / ffmpeg / tesseract (no new pyproject deps). +- All scripts support `--help` and a `--json` machine-readable mode for tests. + +### FR5. Per-child pipeline (5 phases) + +Each child track executes: + +| Phase | Tasks | Output | +|---|---|---| +| **1. Acquire** | Run `extract_transcript.py` (always succeeds, fast). Run `download_video.py` if frame extraction needs video. | `transcript.json`, `download.log` | +| **2. Keyframes** | Run `extract_keyframes.py` with sensible defaults (threshold 0.4). Manual review of frame set. | `frames/*.jpg`, `extraction_meta.json` | +| **3. OCR** | Run `ocr_frames.py` on frames. Spot-check OCR quality. | `ocr.md` | +| **4. Synthesis** | Tier 3 worker prompt: transcript + OCR + frame images → report.md (target 1000-10000 LOC). Human review + iteration. | `report.md`, `summary.md` | +| **5. Verification** | Idempotency check (re-run scripts — should not break). Audit checklist. End-of-track report. | `tests/artifacts//` | + +### FR6. Per-video report structure (8 sections, target 1000-10000 LOC) + +Each `report.md` follows this structure (mirrors `nagent_review`/`fable_review` style): + +``` +#