Private
Public Access
conductor(video_analysis_campaign): Initialize umbrella track + 12 child + 1 synthesis scaffold
Pass 1 of 3 user research campaign (12 videos, 5 clusters). - Umbrella: spec.md (full design), plan.md, metadata.json, state.toml, README.md - Multi-pass framing (Pass 2 de-obfuscation, Pass 3 projection) - Lossless preservation directive (1000-10000 LOC per video report target) - Tooling prerequisites: yt-dlp, cv2, imagehash install in repo venv - 5 reusable scripts to live in scripts/video_analysis/ (TDD) - 12 children + 1 synthesis = 14 folders total
This commit is contained in:
@@ -0,0 +1,52 @@
|
||||
# Video Analysis Campaign (2026-06-21)
|
||||
|
||||
**Status:** Active (spec approved 2026-06-21)
|
||||
**Owner:** Tier 1 Orchestrator (umbrella + synthesis spec); Tier 2 Tech Lead (per-child execution)
|
||||
**Type:** Multi-track research campaign (14 folders total)
|
||||
|
||||
This is **Pass 1 of 3** in a research campaign to penetrate the AI field. See [spec.md](./spec.md) §0 for the multi-pass framing and §11 for the Pass 2/3 handoff contracts.
|
||||
|
||||
## Children (in execution order)
|
||||
|
||||
| # | Slug | Title | Cluster | Track Folder | Status |
|
||||
|---|------|-------|---------|--------------|--------|
|
||||
| 1 | `cs229_building_llms` | Stanford CS229 — Building LLMs | E | [video_analysis_cs229_building_llms_20260621/](./video_analysis_cs229_building_llms_20260621/) | [ ] |
|
||||
| 2 | `probability_logic` | Probability Theory is an Extension of Logic | A | [video_analysis_probability_logic_20260621/](./video_analysis_probability_logic_20260621/) | [ ] |
|
||||
| 3 | `entropy_epiplexity` | From Entropy to Epiplexity (Wilson & Finzi) | A | [video_analysis_entropy_epiplexity_20260621/](./video_analysis_entropy_epiplexity_20260621/) | [ ] |
|
||||
| 4 | `score_dynamics_giorgini` | Learning Dynamics from Statistics (Giorgini) | A | [video_analysis_score_dynamics_giorgini_20260621/](./video_analysis_score_dynamics_giorgini_20260621/) | [ ] |
|
||||
| 5 | `platonic_intelligence_kumar` | Towards a Platonic Intelligence (Kumar) | B | [video_analysis_platonic_intelligence_kumar_20260621/](./video_analysis_platonic_intelligence_kumar_20260621/) | [ ] |
|
||||
| 6 | `free_lunches_levin` | Free Lunches (Levin) | B | [video_analysis_free_lunches_levin_20260621/](./video_analysis_free_lunches_levin_20260621/) | [ ] |
|
||||
| 7 | `generic_systems_fields` | Interesting Behavior by Generic Systems (Fields) | C | [video_analysis_generic_systems_fields_20260621/](./video_analysis_generic_systems_fields_20260621/) | [ ] |
|
||||
| 8 | `brain_counterintuitive` | Most Counterintuitive Way to Build a Brain | C | [video_analysis_brain_counterintuitive_20260621/](./video_analysis_brain_counterintuitive_20260621/) | [ ] |
|
||||
| 9 | `neural_dynamics_miller` | Cognition Emerges from Neural Dynamics (Miller) | C | [video_analysis_neural_dynamics_miller_20260621/](./video_analysis_neural_dynamics_miller_20260621/) | [ ] |
|
||||
| 10 | `multiscale_hoffman` | Multiscale Logic of Collective Intelligence (Hoffman & Prakash) | C | [video_analysis_multiscale_hoffman_20260621/](./video_analysis_multiscale_hoffman_20260621/) | [ ] |
|
||||
| 11 | `cs336_architectures` | Stanford CS336 Lecture 3: Architectures | E | [video_analysis_cs336_architectures_20260621/](./video_analysis_cs336_architectures_20260621/) | [ ] |
|
||||
| 12 | `creikey_dl_cv` | Creikey — DL/CV for Game Developers | D | [video_analysis_creikey_dl_cv_20260621/](./video_analysis_creikey_dl_cv_20260621/) | [ ] |
|
||||
|
||||
## Cross-cutting
|
||||
|
||||
| | Track | Status |
|
||||
|---|-------|--------|
|
||||
| Synthesis (blocked by all 12) | [video_analysis_synthesis_20260621/](./video_analysis_synthesis_20260621/) | [ ] |
|
||||
|
||||
## Status legend
|
||||
|
||||
- `[ ]` — not started
|
||||
- `[~]` — in progress
|
||||
- `[x]` — shipped
|
||||
- `[!]` — blocked
|
||||
|
||||
## Cluster legend
|
||||
|
||||
- **A** — Math & information-theoretic foundations (3 videos)
|
||||
- **B** — Platonic / geometric AI representations (2 videos)
|
||||
- **C** — Biological / cognitive / generic systems (4 videos)
|
||||
- **D** — Applied / practical (1 video)
|
||||
- **E** — Stanford course VODs >1hr (2 videos)
|
||||
|
||||
## See also
|
||||
|
||||
- [spec.md](./spec.md) — full design (Overview, Current State Audit, Goals, FRs, NFRs, Architecture, Future-Pass Hooks, Risk Register, User Directives)
|
||||
- [plan.md](./plan.md) — campaign-level plan (Phases 0-4)
|
||||
- [metadata.json](./metadata.json) — scope, verification criteria, risk register
|
||||
- [state.toml](./state.toml) — current phase + task tracking
|
||||
@@ -0,0 +1,231 @@
|
||||
{
|
||||
"track_id": "video_analysis_campaign_20260621",
|
||||
"name": "Video Analysis Campaign (12 videos, 5 clusters, 3 passes)",
|
||||
"created": "2026-06-21",
|
||||
"status": "spec_approved",
|
||||
"blocked_by": [],
|
||||
"blocks": [
|
||||
"video_analysis_synthesis_20260621"
|
||||
],
|
||||
"priority": "A",
|
||||
"rationale": "User-blocking research campaign to extract foundational knowledge from 12 curated YouTube videos on AI inference, ML, biological learning, and neuro-compute. The artifacts feed future Pass 2 (de-obfuscation via user's math encoding) and Pass 3 (projection to applied domain). Lossless preservation is the design priority. Research-only — no src/ changes, no new pyproject deps (all tools via subprocess or existing venv).",
|
||||
"type": "multi-track research campaign (1 umbrella + 12 child tracks + 1 synthesis = 14 folders total)",
|
||||
"domain": "meta-tooling (research artifacts; no manual_slop src/ changes)",
|
||||
"scope": {
|
||||
"new_folders": [
|
||||
"conductor/tracks/video_analysis_campaign_20260621/",
|
||||
"conductor/tracks/video_analysis_cs229_building_llms_20260621/",
|
||||
"conductor/tracks/video_analysis_probability_logic_20260621/",
|
||||
"conductor/tracks/video_analysis_entropy_epiplexity_20260621/",
|
||||
"conductor/tracks/video_analysis_score_dynamics_giorgini_20260621/",
|
||||
"conductor/tracks/video_analysis_platonic_intelligence_kumar_20260621/",
|
||||
"conductor/tracks/video_analysis_free_lunches_levin_20260621/",
|
||||
"conductor/tracks/video_analysis_generic_systems_fields_20260621/",
|
||||
"conductor/tracks/video_analysis_brain_counterintuitive_20260621/",
|
||||
"conductor/tracks/video_analysis_neural_dynamics_miller_20260621/",
|
||||
"conductor/tracks/video_analysis_multiscale_hoffman_20260621/",
|
||||
"conductor/tracks/video_analysis_cs336_architectures_20260621/",
|
||||
"conductor/tracks/video_analysis_creikey_dl_cv_20260621/",
|
||||
"conductor/tracks/video_analysis_synthesis_20260621/"
|
||||
],
|
||||
"new_files_per_child": [
|
||||
"spec.md (lightweight)",
|
||||
"artifacts/transcript.json",
|
||||
"artifacts/ocr.md",
|
||||
"artifacts/frames/<scene>_<ts>.jpg (deduplicated)",
|
||||
"artifacts/extraction_meta.json",
|
||||
"report.md (1000-10000 LOC target)",
|
||||
"summary.md (200-400 words)"
|
||||
],
|
||||
"new_files_scripts": [
|
||||
"scripts/video_analysis/download_video.py",
|
||||
"scripts/video_analysis/extract_transcript.py",
|
||||
"scripts/video_analysis/extract_keyframes.py",
|
||||
"scripts/video_analysis/ocr_frames.py",
|
||||
"scripts/video_analysis/synthesize_report.py"
|
||||
],
|
||||
"new_files_tests": [
|
||||
"tests/test_video_analysis_download_video.py",
|
||||
"tests/test_video_analysis_extract_transcript.py",
|
||||
"tests/test_video_analysis_extract_keyframes.py",
|
||||
"tests/test_video_analysis_ocr_frames.py",
|
||||
"tests/test_video_analysis_synthesize_report.py"
|
||||
],
|
||||
"new_files_synthesis": [
|
||||
"conductor/tracks/video_analysis_synthesis_20260621/spec.md",
|
||||
"conductor/tracks/video_analysis_synthesis_20260621/per_video_summary.md",
|
||||
"conductor/tracks/video_analysis_synthesis_20260621/report.md"
|
||||
],
|
||||
"modified_files": [],
|
||||
"deleted_files": [],
|
||||
"gitignored_patterns": [
|
||||
"*.mp4 (video files - too large for git)",
|
||||
"artifacts/frames/*.jpg if >500KB each",
|
||||
"tests/artifacts/<slug>/ (per AGENTS.md artifact isolation)"
|
||||
]
|
||||
},
|
||||
"estimated_effort": {
|
||||
"method": "scope (per conductor/workflow.md Tier 1 Track Initialization Rules). NO day estimates.",
|
||||
"phase_0": "4 tasks: tooling prerequisites (yt-dlp, cv2, imagehash, OCR backend decision)",
|
||||
"phase_1": "10 tasks: 5 reusable scripts with TDD (red + green per script)",
|
||||
"phase_2": "12 child tracks × 5 phases each = 60 child track execution tasks (tracked in child plans, not this umbrella)",
|
||||
"phase_3": "1 synthesis track (blocked by all 12 children)",
|
||||
"phase_4": "4 tasks: campaign closeout (README update, end-of-track report, archive move, chronology update)",
|
||||
"summary": "14 track folders (1 umbrella + 12 children + 1 synthesis), 5 reusable scripts, ~40-60 unit tests, 12 reports (1000-10000 LOC each), 12 summaries, 1 cross-cutting synthesis report. No day estimates per project convention."
|
||||
},
|
||||
"verification_criteria": [
|
||||
"yt-dlp installed and importable in this repo's venv",
|
||||
"cv2, imagehash, PIL installed in this repo's venv",
|
||||
"OCR backend chosen (winsdk or tesseract) and working",
|
||||
"All 5 scripts in scripts/video_analysis/ have passing TDD tests",
|
||||
"All 12 child tracks shipped: each has transcript.json, frames/, ocr.md, report.md (1000-10000 LOC), summary.md",
|
||||
"Synthesis track shipped: per_video_summary.md + report.md",
|
||||
"Umbrella README.md shows all 12 children + synthesis as shipped",
|
||||
"End-of-track report at docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md",
|
||||
"All artifacts preserved losslessly (JSON for transcripts, raw images for frames, plain text for OCR)",
|
||||
"No src/*.py files created or modified (per AGENTS.md File Size and Naming Convention)",
|
||||
"No new pyproject.toml dependencies (all tools via subprocess or existing venv)",
|
||||
"Future-pass hooks (§11 of spec.md) intact and documented for Pass 2/3"
|
||||
],
|
||||
"risk_register": [
|
||||
{
|
||||
"id": "R1",
|
||||
"title": "yt-dlp not installed locally",
|
||||
"likelihood": "high",
|
||||
"scope_impact": "First child track blocked until installed",
|
||||
"mitigation": "Install via pip install yt-dlp in this repo's venv (single one-time task in Phase 0)"
|
||||
},
|
||||
{
|
||||
"id": "R2",
|
||||
"title": "OCR quality insufficient for technical content",
|
||||
"likelihood": "medium",
|
||||
"scope_impact": "Some frames may have illegible text",
|
||||
"mitigation": "Spot-check OCR per frame; manually transcribe critical frames in the report.md section"
|
||||
},
|
||||
{
|
||||
"id": "R3",
|
||||
"title": "Report exceeds 10000 LOC target",
|
||||
"likelihood": "low",
|
||||
"scope_impact": "User may want to split",
|
||||
"mitigation": "Pass 2 can split; Pass 1 should not artificially cap"
|
||||
},
|
||||
{
|
||||
"id": "R4",
|
||||
"title": "Video mp4 files exceed disk space",
|
||||
"likelihood": "medium",
|
||||
"scope_impact": "Could hit quota",
|
||||
"mitigation": "Delete mp4 after frame extraction (extract_keyframes.py should do this)"
|
||||
},
|
||||
{
|
||||
"id": "R5",
|
||||
"title": "Two videos failed oEmbed fetch",
|
||||
"likelihood": "confirmed for 9vM4p9NN0Ts and lVynu4bo1rY",
|
||||
"scope_impact": "Unknown until track execution",
|
||||
"mitigation": "User confirmed identities. yt-dlp may still work (different from oEmbed). Verify in Phase 1 of each track."
|
||||
},
|
||||
{
|
||||
"id": "R6",
|
||||
"title": "User's math encoding notation (Pass 2) lost",
|
||||
"likelihood": "medium",
|
||||
"scope_impact": "Blocks Pass 2",
|
||||
"mitigation": "User action item: rediscover/redefine encoding before Pass 2 starts. Recorded in spec.md §11.1."
|
||||
},
|
||||
{
|
||||
"id": "R7",
|
||||
"title": "Pass 1 over-summarization loses signal for Pass 2",
|
||||
"likelihood": "medium (if not enforced)",
|
||||
"scope_impact": "Cascades to Pass 2/3",
|
||||
"mitigation": "The 1000-10000 LOC target + spec.md §0 explicit warning + per-section completeness check in verification"
|
||||
},
|
||||
{
|
||||
"id": "R8",
|
||||
"title": "Tier 2 capacity for 12+ child tracks",
|
||||
"likelihood": "medium",
|
||||
"scope_impact": "Tracks ship in sequence",
|
||||
"mitigation": "Each child is independently shippable; the campaign is async"
|
||||
},
|
||||
{
|
||||
"id": "R9",
|
||||
"title": "Transcript API rate-limiting",
|
||||
"likelihood": "low",
|
||||
"scope_impact": "Some videos may fail on first fetch",
|
||||
"mitigation": "Retry with backoff in extract_transcript.py"
|
||||
},
|
||||
{
|
||||
"id": "R10",
|
||||
"title": "cv2 / imagehash not in this repo's venv",
|
||||
"likelihood": "high (verified - exist only in foreign venvs)",
|
||||
"scope_impact": "Blocks keyframe extraction",
|
||||
"mitigation": "Install via pip install opencv-python imagehash pillow in this repo's venv (single one-time task in Phase 0)"
|
||||
}
|
||||
],
|
||||
"architecture_reference": {
|
||||
"primary_documents": [
|
||||
"conductor/workflow.md (track convention, per-task commits, git notes, verification protocol)",
|
||||
"conductor/code_styleguides/python.md (1-space indent, type hints, no comments)",
|
||||
"conductor/code_styleguides/error_handling.md (Result[T] pattern for new scripts)",
|
||||
"AGENTS.md (artifact isolation, file naming, no new src/<thing>.py)",
|
||||
"conductor/chronology.md (after campaign ships, 14 new rows added here)"
|
||||
],
|
||||
"related_tracks": [
|
||||
"conductor/tracks/intent_dsl_survey_20260612/ (Pass 2 may build on this)",
|
||||
"conductor/tracks/nagent_review_20260608/ (precedent for deep-dive report format)",
|
||||
"conductor/tracks/fable_review_20260617/ (precedent for synthesis report format)",
|
||||
"conductor/tracks/chronology_20260619/ (precedent for spec/plan/metadata/state schema)"
|
||||
],
|
||||
"external_references": [
|
||||
"C:/projects/forth/bootslop/download_videos.py (yt-dlp usage reference)",
|
||||
"C:/projects/forth/bootslop/extract_frames.py (imagehash + cv2 keyframe extraction reference)",
|
||||
"C:/projects/forth/bootslop/process_visuals.py (winsdk OCR + visual analysis reference)",
|
||||
"C:/projects/forth/bootslop/ocr_interaction.py (standalone OCR reference)",
|
||||
"https://pypi.org/project/youtube-transcript-api/",
|
||||
"C:/projects/kasa/venv/Lib/site-packages/cv2/opencv_videoio_ffmpeg481_64.dll (proves cv2/ffmpeg installs on this machine)"
|
||||
],
|
||||
"styleguides_applied": [
|
||||
"data_oriented_design.md (referenced by Pass 3, not directly by Pass 1)",
|
||||
"python.md (1-space indent for all new scripts)",
|
||||
"error_handling.md (Result[T] for all new scripts)",
|
||||
"feature_flags.md (scripts are file-presence, no config flags needed)",
|
||||
"workspace_paths.md (test artifacts in tests/artifacts/)"
|
||||
]
|
||||
},
|
||||
"deferred_to_followup_tracks": [
|
||||
{
|
||||
"title": "Pass 2: De-obfuscation via user's math encoding notation",
|
||||
"description": "Apply the user's custom math encoding/compression notation to reduce DSL + niche math notation/verbiage into something the user can understand. Consumes all Pass 1 artifacts.",
|
||||
"track_status": "not started - blocked by this track",
|
||||
"blocker_action_item": "User must rediscover/redefine their 'compress/decompress math info' encoding notation before Pass 2 starts. See spec.md §11.1."
|
||||
},
|
||||
{
|
||||
"title": "Pass 3: Projection to user's applied domain",
|
||||
"description": "Apply Pass 2 outputs to user's preferred code style. Influences: handmade/data-oriented/GPGPU community (Lottes, Onat, Jebrim) + user's own caveats.",
|
||||
"track_status": "not started - blocked by Pass 2",
|
||||
"blocker_action_item": "User must articulate 'own caveats' before Pass 3 starts. See spec.md §11.2."
|
||||
}
|
||||
],
|
||||
"regressions_and_pre_existing_failures": [],
|
||||
"pre_existing_failures_remaining": [],
|
||||
"user_directives": [
|
||||
"Order confirmed (12-video sequence, 2026-06-21)",
|
||||
"Report target: minimum 1000 LOC, maximum 10000 LOC markdown per video (2026-06-21)",
|
||||
"Multi-pass framing: Pass 1 = information extraction (this track), Pass 2 = de-obfuscation, Pass 3 = projection (2026-06-21)",
|
||||
"Pass 1 artifacts must be lossless - over-summarization is data loss for Pass 2 (2026-06-21)",
|
||||
"Stanford CS229 = 9vM4p9NN0Ts, Stanford CS336 Lecture 3 = lVynu4bo1rY (user-confirmed mapping, 2026-06-21)",
|
||||
"Future-pass hooks must be explicit in spec.md so the next agent / future-self can pick up the thread (2026-06-21)",
|
||||
"No day estimates per conductor/workflow.md Tier 1 Track Initialization Rules (added 2026-06-16). Scope measured in files/sites only."
|
||||
],
|
||||
"videos": [
|
||||
{"order": 1, "slug": "cs229_building_llms", "cluster": "E", "title": "Stanford CS229 - Building Large Language Models (LLMs)", "youtube_id": "9vM4p9NN0Ts", "author": "Stanford CS229"},
|
||||
{"order": 2, "slug": "probability_logic", "cluster": "A", "title": "Probability Theory is an Extension of Logic", "youtube_id": "0yF9TvMeAzM", "author": null},
|
||||
{"order": 3, "slug": "entropy_epiplexity", "cluster": "A", "title": "From Entropy to Epiplexity", "youtube_id": "_U8AwUq_aJQ", "author": "Andrew Wilson and Marc Finzi"},
|
||||
{"order": 4, "slug": "score_dynamics_giorgini", "cluster": "A", "title": "Learning Dynamics from Statistics: a score-based approach", "youtube_id": "P75iVMmbqQk", "author": "Ludovico Giorgini"},
|
||||
{"order": 5, "slug": "platonic_intelligence_kumar", "cluster": "B", "title": "Towards a Platonic Intelligence with Unified Factored Representations", "youtube_id": "1mXUFweWOug", "author": "Akarsh Kumar"},
|
||||
{"order": 6, "slug": "free_lunches_levin", "cluster": "B", "title": "Free Lunches: Model Systems for Studying the Agential Gifts from the Platonic Space", "youtube_id": "K8BmMU1Tm-I", "author": "Michael Levin"},
|
||||
{"order": 7, "slug": "generic_systems_fields", "cluster": "C", "title": "Interesting Behavior by Generic Systems", "youtube_id": "QeMajYvhEbI", "author": "Chris Fields"},
|
||||
{"order": 8, "slug": "brain_counterintuitive", "cluster": "C", "title": "The Most Counterintuitive Way to Build a Brain", "youtube_id": "cDxtFtoQVNc", "author": null},
|
||||
{"order": 9, "slug": "neural_dynamics_miller", "cluster": "C", "title": "Cognition Emerges from Neural Dynamics", "youtube_id": "0BS-BzEFTXA", "author": "Earl Miller"},
|
||||
{"order": 10, "slug": "multiscale_hoffman", "cluster": "C", "title": "A Multiscale Logic of Collective Intelligence", "youtube_id": "YnfaT5APPB0", "author": "Donald Hoffman and Chetan Prakash"},
|
||||
{"order": 11, "slug": "cs336_architectures", "cluster": "E", "title": "Stanford CS336 Lecture 3: Architectures", "youtube_id": "lVynu4bo1rY", "author": "Stanford CS336 Spring 2026"},
|
||||
{"order": 12, "slug": "creikey_dl_cv", "cluster": "D", "title": "Creikey - Deep Learning and Computer Vision for Game Developers (BSC 2025)", "youtube_id": "yxkUvXs-hoQ", "author": "Creikey"}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
# Plan: Video Analysis Campaign (umbrella)
|
||||
|
||||
This is the umbrella-level plan. Per the Tier 1 Track Initialization Rules, scope is measured in files/sites — no day estimates.
|
||||
|
||||
## Phase 0: Tooling Prerequisites
|
||||
|
||||
Must be completed before any child track can ship. One-time setup.
|
||||
|
||||
- [ ] **Task 0.1:** Install `yt-dlp` in this repo's venv (`pip install yt-dlp`). Verify with `python -c "import yt_dlp; print(yt_dlp.version.__version__)"`.
|
||||
- [ ] **Task 0.2:** Install `opencv-python`, `imagehash`, `pillow` in this repo's venv. Verify imports.
|
||||
- [ ] **Task 0.3:** Decide on OCR backend. Try `winsdk` first (matches bootslop); fall back to `tesseract` if `winsdk` proves problematic.
|
||||
- [ ] **Task 0.4:** Create `scripts/video_analysis/` namespace and `tests/test_video_analysis_*.py` skeleton.
|
||||
|
||||
## Phase 1: Reusable Tooling (5 scripts, TDD)
|
||||
|
||||
Each script is independently TDD-tested. Order: extract_transcript → download_video → extract_keyframes → ocr_frames → synthesize_report (synthesize_report is last because it composes the others).
|
||||
|
||||
- [ ] **Task 1.1:** Write tests for `extract_transcript.py` (red). Tests cover: success path, network error, missing video ID, malformed JSON response, retry behavior.
|
||||
- [ ] **Task 1.2:** Implement `extract_transcript.py` (green). CLI: `--url`, `--output`, `--json`. Outputs `transcript.json` with `segments` (list of `{start, duration, text}`) + `plain` (joined text) + `metadata` (video ID, fetch timestamp).
|
||||
- [ ] **Task 1.3:** Write tests for `download_video.py` (red).
|
||||
- [ ] **Task 1.4:** Implement `download_video.py` (green). CLI: `--url`, `--output`, `--json`. Subprocess `yt-dlp`. Outputs `download.log`.
|
||||
- [ ] **Task 1.5:** Write tests for `extract_keyframes.py` (red).
|
||||
- [ ] **Task 1.6:** Implement `extract_keyframes.py` (green). CLI: `--video`, `--output-dir`, `--threshold`, `--json`. Uses ffmpeg `select=gt(scene\,0.4)` + cv2 frame extraction + imagehash dedup. Outputs `frames/*.jpg` + `extraction_meta.json`.
|
||||
- [ ] **Task 1.7:** Write tests for `ocr_frames.py` (red).
|
||||
- [ ] **Task 1.8:** Implement `ocr_frames.py` (green). CLI: `--frames-dir`, `--output`, `--json`. Uses winsdk (with tesseract fallback). Outputs `ocr.md` with one section per frame (image path + OCR text + timestamp).
|
||||
- [ ] **Task 1.9:** Write tests for `synthesize_report.py` (red).
|
||||
- [ ] **Task 1.10:** Implement `synthesize_report.py` (green). CLI: `--url`, `--slug`, `--output-dir`, `--json`. Orchestrates the full pipeline for one video. Outputs `artifacts/` populated + `report.md` stub + `summary.md` stub.
|
||||
|
||||
## Phase 2: Per-Child Tracks (12 tracks)
|
||||
|
||||
Each child track follows the 5-phase pipeline. The Tier 2 Tech Lead executes these. The umbrella plan does NOT enumerate per-child tasks — those live in each child's `plan.md` (created during execution).
|
||||
|
||||
**Execution order** (per §6 of spec.md):
|
||||
|
||||
| # | Slug | Cluster | Notes |
|
||||
|---|------|---------|-------|
|
||||
| 1 | `cs229_building_llms` | E | Verify yt-dlp access (oEmbed failed 401) |
|
||||
| 2 | `probability_logic` | A | |
|
||||
| 3 | `entropy_epiplexity` | A | |
|
||||
| 4 | `score_dynamics_giorgini` | A | |
|
||||
| 5 | `platonic_intelligence_kumar` | B | |
|
||||
| 6 | `free_lunches_levin` | B | |
|
||||
| 7 | `generic_systems_fields` | C | |
|
||||
| 8 | `brain_counterintuitive` | C | |
|
||||
| 9 | `neural_dynamics_miller` | C | |
|
||||
| 10 | `multiscale_hoffman` | C | |
|
||||
| 11 | `cs336_architectures` | E | Verify yt-dlp access (oEmbed failed 401) |
|
||||
| 12 | `creikey_dl_cv` | D | |
|
||||
|
||||
**Note on E-cluster yt-dlp verification:** the oEmbed API returned 401 for `9vM4p9NN0Ts` and `lVynu4bo1rY`. This may be an oEmbed-specific restriction (private/age-restricted); `yt-dlp` may still work. Phase 1 of those child tracks must verify yt-dlp access before downloading anything.
|
||||
|
||||
## Phase 3: Synthesis Track (blocked by all 12 children)
|
||||
|
||||
After all 12 child tracks ship, the synthesis track consumes their outputs and produces:
|
||||
- `per_video_summary.md` (the "summary of each video" the user requested)
|
||||
- `report.md` (the "summary report of key takeaways" — theme matrix, concept map, top takeaways, math prereq graph, open questions, next-watch list)
|
||||
|
||||
## Phase 4: Campaign Closeout
|
||||
|
||||
- [ ] **Task 4.1:** Update umbrella `README.md` with final statuses (all 12 children + synthesis shipped).
|
||||
- [ ] **Task 4.2:** Write end-of-track report at `docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md`.
|
||||
- [ ] **Task 4.3:** Move umbrella + 13 children to `conductor/archive/` per the project's archiving convention.
|
||||
- [ ] **Task 4.4:** Update `conductor/chronology.md` with the 14 track rows.
|
||||
|
||||
## Verification (gate per workflow.md)
|
||||
|
||||
Each child track's Phase 5 must include:
|
||||
- [ ] Idempotency check: re-running scripts produces identical outputs (modulo timestamps)
|
||||
- [ ] Audit checklist: every section of the report is populated, no "TBD"
|
||||
- [ ] Tests pass
|
||||
- [ ] Per-task commits with git notes
|
||||
|
||||
The campaign-level verification (Phase 4.2 end-of-track report) confirms all 12 children + 1 synthesis are shipped and the future-pass hooks (§11 of spec.md) are intact.
|
||||
@@ -0,0 +1,439 @@
|
||||
# Track Specification: Video Analysis Campaign (2026-06-21)
|
||||
|
||||
**Status:** Active (spec approved 2026-06-21)
|
||||
**Initialized:** 2026-06-21
|
||||
**Owner:** Tier 1 Orchestrator (umbrella spec/synthesis); Tier 2 Tech Lead (per-child execution)
|
||||
**Priority:** A (user-blocking; long-running research campaign)
|
||||
**Type:** Multi-track research campaign (1 umbrella + 12 child tracks + 1 synthesis = 14 folders)
|
||||
**Domain:** Meta-tooling (produces research artifacts; no `src/` changes to manual_slop)
|
||||
|
||||
> **Purpose.** This umbrella track organizes a 12-video research campaign to extract foundational knowledge from the user's preferred sources on AI inference, ML, biological learning, and neuro-compute. The artifacts (transcripts, keyframes, OCR, deep-dive reports) are intermediate inputs to future campaign passes (Pass 2: de-obfuscation; Pass 3: projection to applied domain). **Pass 1 is information extraction + distillation; lossless preservation is the design priority.**
|
||||
|
||||
> **Companion docs.** This spec is the umbrella. The per-video spec is at `conductor/tracks/video_analysis_<slug>_20260621/spec.md` (one per child, 12 total). The cross-cutting synthesis spec is at `conductor/tracks/video_analysis_synthesis_20260621/spec.md`.
|
||||
|
||||
---
|
||||
|
||||
## 0. Campaign Context (multi-pass framing — load-bearing)
|
||||
|
||||
This is **Pass 1 of 3** in a long-running research campaign.
|
||||
|
||||
| Pass | Goal | Status | Dependencies |
|
||||
|---|---|---|---|
|
||||
| **1 (THIS)** | Information extraction + distillation. Raw transcripts, keyframes, OCR, deep-dive reports per video. Foundational knowledge base. | Active (this track). | None. |
|
||||
| **2 (FUTURE)** | De-obfuscation via user's custom math encoding notation. Reduce DSL + niche math notation/verbiage into something the user (and associates) can understand. | Not started. **User must rediscover/redefine their encoding system before starting** ("compress/decompress math info" — they have a "handmade" notation from prior work but need to find it). Related: `intent_dsl_survey_20260612`, DSL patterns in `conductor/` docs + track reports. | Blocked by Pass 1. |
|
||||
| **3 (FUTURE)** | Projection to user's applied domain. Apply learnings to user's preferred code style. Influences: handmade / data-oriented / GPGPU community (Timothy Lottes, Onat Türkçüoğlu, Jebrim) + user's own caveats. Some preferences already in `conductor/workflow.md` (data-oriented design styleguide). | Not started. | Blocked by Pass 2. |
|
||||
|
||||
**Implication for Pass 1 artifacts (load-bearing — read carefully):**
|
||||
- **Raw data MUST be preserved in lossless form.** JSON for transcripts (timestamped), raw images for frames, plain text for OCR. Pass 2 needs every signal.
|
||||
- **Reports should be DETAILED, not summarized.** Per the user directive (2026-06-21), the target is **1000-10000 LOC of markdown per video report**. Over-summarization here is data loss for later.
|
||||
- **Synthesis report preserves detail too.** Pass 2 will compress.
|
||||
- **Don't optimize for "pretty" at the cost of "complete."**
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
This campaign extracts deep knowledge from 12 YouTube videos the user has curated, organized into 5 thematic clusters:
|
||||
|
||||
- **Cluster E — Stanford course VODs (>1hr each):** 2 videos
|
||||
- `9vM4p9NN0Ts` — Stanford CS229 — Machine Learning — Building Large Language Models (LLMs)
|
||||
- `lVynu4bo1rY` — Stanford CS336 — Language Modeling from Scratch, Spring 2026, Lecture 3: Architectures
|
||||
- **Cluster A — Math & information-theoretic foundations:** 3 videos
|
||||
- `0yF9TvMeAzM` — Probability Theory is an Extension of Logic
|
||||
- `_U8AwUq_aJQ` — "From Entropy to Epiplexity" (Andrew Wilson and Marc Finzi)
|
||||
- `P75iVMmbqQk` — "Learning Dynamics from Statistics: a score-based approach" (Ludovico Giorgini)
|
||||
- **Cluster B — Platonic / geometric AI representations:** 2 videos
|
||||
- `1mXUFweWOug` — "Towards a Platonic Intelligence with Unified Factored Representations" (Akarsh Kumar)
|
||||
- `K8BmMU1Tm-I` — "Free Lunches: Model Systems for Studying the Agential Gifts from the Platonic Space" (Michael Levin)
|
||||
- **Cluster C — Biological / cognitive / generic systems:** 4 videos
|
||||
- `cDxtFtoQVNc` — The Most Counterintuitive Way to Build a Brain
|
||||
- `YnfaT5APPB0` — "A Multiscale Logic of Collective Intelligence" (Donald Hoffman and Chetan Prakash)
|
||||
- `0BS-BzEFTXA` — "Cognition Emerges from Neural Dynamics" (Earl Miller)
|
||||
- `QeMajYvhEbI` — "Interesting Behavior by Generic Systems" (Chris Fields)
|
||||
- **Cluster D — Applied / practical:** 1 video
|
||||
- `yxkUvXs-hoQ` — Creikey — Deep Learning and Computer Vision for Game Developers (BSC 2025)
|
||||
|
||||
**Total: 12 videos across 5 clusters.**
|
||||
|
||||
The campaign delivers:
|
||||
- Per-video: transcript + keyframes + OCR + deep-dive report (1000-10000 LOC markdown each) + summary
|
||||
- Cross-cutting: per-video roll-up + synthesis report with theme matrix, concept map, top takeaways, math prerequisite graph, open questions, and recommended next-watch list
|
||||
|
||||
---
|
||||
|
||||
## 2. Current State Audit (as of 2026-06-21)
|
||||
|
||||
### 2.1 Already Available (DO NOT re-build)
|
||||
|
||||
| Asset | Location | Status |
|
||||
|---|---|---|
|
||||
| `yt-dlp` (Python module) | NOT installed (system `yt-dlp` binary NOT on PATH either) | **BLOCKER.** Must be installed before any track ships. |
|
||||
| `ffmpeg` 8.1.1 | System PATH | Available. |
|
||||
| `youtube-transcript-api` | Python module | Installed and importable. |
|
||||
| `cv2` (opencv-python) with ffmpeg bindings | `C:\projects\kasa\venv\Lib\site-packages\cv2\` (foreign venv; **DO NOT activate**) | Available — need to install in this repo's venv before any track ships. |
|
||||
| `imagehash`, `PIL` | Foreign venvs only | Need to install in this repo's venv. |
|
||||
| `winsdk` (Windows OCR) | Used by bootslop (`C:\projects\forth\bootslop\process_visuals.py`) | Windows-only; not installed here yet. |
|
||||
| `tesseract` (cross-platform OCR fallback) | Not installed | Optional fallback if `winsdk` proves problematic. |
|
||||
| Reference scripts | `C:\projects\forth\bootslop\download_videos.py`, `extract_frames.py`, `process_visuals.py`, `ocr_interaction.py`, `fetch_blog.py`, `fetch_notes.py` | **Reference only.** New scripts will live in `scripts/video_analysis/` (this repo, per AGENTS.md namespace convention). |
|
||||
| Manual Slop's track convention | `conductor/workflow.md`, `conductor/tracks.md`, nagent/fable/chronology precedents | Established. |
|
||||
| Manual Slop's data-oriented styleguide | `conductor/code_styleguides/data_oriented_design.md` | Referenced by Pass 3 (out of scope here). |
|
||||
| Manual Slop's error-handling convention | `conductor/code_styleguides/error_handling.md` (Result[T] pattern) | Applies to any new Python in `scripts/video_analysis/`. |
|
||||
|
||||
### 2.2 Gaps to Fill (this track's scope)
|
||||
|
||||
| # | Gap | Resolution |
|
||||
|---|---|---|
|
||||
| G1 | No reusable scripts for video download / transcript extraction / keyframe extraction / OCR / report synthesis | Create `scripts/video_analysis/` namespace with 5 scripts |
|
||||
| G2 | No tests for the new scripts | TDD: `tests/test_video_analysis_*.py` (~40-60 tests) |
|
||||
| G3 | No per-video deep-dive reports | 12 child tracks, each producing one `report.md` (1000-10000 LOC) + `summary.md` (200-400 words) |
|
||||
| G4 | No cross-cutting synthesis | 1 synthesis track, blocked by all 12 children, producing `per_video_summary.md` + `report.md` |
|
||||
| G5 | No campaign-level index | `README.md` at umbrella folder with one row per child + status |
|
||||
| G6 | No transcripts/frames/OCR artifacts | Created per-child under `artifacts/` (lossless JSON + raw images) |
|
||||
| G7 | Future-pass hooks not documented | This spec §11 explicitly records the Pass 2/3 dependencies so the next agent can pick up the thread |
|
||||
|
||||
---
|
||||
|
||||
## 3. Goals
|
||||
|
||||
1. **Lossless extraction.** Every signal from the 12 videos (spoken word, on-screen text, keyframes) is captured in a machine-readable form. Pass 2 has all the raw material.
|
||||
2. **Per-video deep understanding.** Each video gets a 1000-10000 LOC deep-dive report covering: TL;DR, key concepts, frame analysis, transcript highlights, math/theoretical content, cross-video connections, open questions, references.
|
||||
3. **Cross-cutting synthesis.** A campaign-level report maps themes across the 5 clusters, links concepts between videos, surfaces 5-10 high-level takeaways, and recommends a next-watch list.
|
||||
4. **Reusable tooling.** The 5 scripts in `scripts/video_analysis/` are independently TDD-tested and usable for any future video analysis (Pass 2, Pass 3, ad-hoc).
|
||||
5. **No manual_slop `src/` changes.** This is a research campaign; the deliverable is the artifacts and reports.
|
||||
6. **Future-pass documentation.** This spec records the Pass 2/3 dependencies so the next agent (or the user, after context compaction) has a clear handoff.
|
||||
|
||||
---
|
||||
|
||||
## 4. Functional Requirements
|
||||
|
||||
### FR1. Umbrella folder + README
|
||||
|
||||
**WHERE:** New folder `conductor/tracks/video_analysis_campaign_20260621/`.
|
||||
|
||||
**WHAT:** The umbrella folder contains:
|
||||
- `spec.md` (this file)
|
||||
- `plan.md` (campaign-level plan — pointers to children)
|
||||
- `metadata.json` (campaign metadata)
|
||||
- `state.toml` (campaign state)
|
||||
- `README.md` (one row per child + status — like a mini-chronology for the campaign)
|
||||
|
||||
**The README structure:**
|
||||
```markdown
|
||||
# Video Analysis Campaign
|
||||
|
||||
## Children (in execution order)
|
||||
|
||||
| # | Slug | Title | Cluster | Track Folder | Status |
|
||||
|---|------|-------|---------|--------------|--------|
|
||||
| 1 | cs229_building_llms | CS229 — Building LLMs | E | [tracks/video_analysis_cs229_building_llms_20260621/](./video_analysis_cs229_building_llms_20260621/) | [~] |
|
||||
| ... |
|
||||
|
||||
## Cross-cutting
|
||||
|
||||
| | Track | Status |
|
||||
|---|-------|--------|
|
||||
| Synthesis | [tracks/video_analysis_synthesis_20260621/](./video_analysis_synthesis_20260621/) | [ ] (blocked by all 12) |
|
||||
```
|
||||
|
||||
### FR2. 12 child track folders (one per video)
|
||||
|
||||
**WHERE:** New folders `conductor/tracks/video_analysis_<slug>_20260621/` (12 total).
|
||||
|
||||
**WHAT:** Each child folder contains at minimum:
|
||||
- `spec.md` (lightweight — references umbrella, lists the video, specifies what to produce, target LOC)
|
||||
- `artifacts/` (created during execution):
|
||||
- `transcript.json` (timestamped segments + plain text)
|
||||
- `download.log` (yt-dlp log if mp4 downloaded)
|
||||
- `frames/<scene>_<ts>.jpg` (deduplicated unique frames)
|
||||
- `ocr.md` (full OCR text per frame)
|
||||
- `report.md` (created during execution — 1000-10000 LOC target)
|
||||
- `summary.md` (created during execution — 200-400 words)
|
||||
|
||||
**Optional (added during execution):** `plan.md`, `metadata.json`, `state.toml` per the standard track convention.
|
||||
|
||||
**Slug convention:** `<descriptive_lowercase_underscore>` — see `slug_to_url` mapping in §7.
|
||||
|
||||
### FR3. 1 synthesis track folder
|
||||
|
||||
**WHERE:** New folder `conductor/tracks/video_analysis_synthesis_20260621/`.
|
||||
|
||||
**WHAT:** Contains:
|
||||
- `spec.md` (lightweight — references umbrella, lists the 12 inputs, specifies the synthesis structure)
|
||||
- `per_video_summary.md` (created during execution — one paragraph per video, the "summary of each video" the user requested)
|
||||
- `report.md` (created during execution — the "summary report of key takeaways")
|
||||
|
||||
**`blocked_by`:** all 12 child tracks (per `state.toml`).
|
||||
|
||||
### FR4. Reusable tooling (5 scripts in `scripts/video_analysis/`)
|
||||
|
||||
Per AGENTS.md: scripts are namespace-isolated by directory. New namespace `scripts/video_analysis/`.
|
||||
|
||||
| Script | Purpose | Inputs | Outputs |
|
||||
|---|---|---|---|
|
||||
| `scripts/video_analysis/download_video.py` | yt-dlp wrapper (subprocess — no new pyproject deps) | video URL, output path | mp4 file at output path + `download.log` |
|
||||
| `scripts/video_analysis/extract_transcript.py` | youtube-transcript-api wrapper | video URL or ID | `transcript.json` (segments + plain) |
|
||||
| `scripts/video_analysis/extract_keyframes.py` | ffmpeg `select=gt(scene\,0.4)` + cv2 + imagehash dedup | mp4 path, output dir, threshold | `frames/*.jpg` + `extraction_meta.json` |
|
||||
| `scripts/video_analysis/ocr_frames.py` | Windows WinSDK OCR (with tesseract fallback) | frames dir | `ocr.md` (one section per frame) |
|
||||
| `scripts/video_analysis/synthesize_report.py` | Orchestrator — runs the full pipeline for one video | video URL, output dir | `artifacts/` populated + `report.md` stub |
|
||||
|
||||
**Conventions:**
|
||||
- All scripts follow `conductor/code_styleguides/error_handling.md` (Result[T] pattern — applies to any new Python in `src/` or `scripts/`).
|
||||
- All scripts follow `conductor/code_styleguides/python.md` (1-space indent, type hints, no comments).
|
||||
- All scripts use `subprocess` for yt-dlp / ffmpeg / tesseract (no new pyproject deps).
|
||||
- All scripts support `--help` and a `--json` machine-readable mode for tests.
|
||||
|
||||
### FR5. Per-child pipeline (5 phases)
|
||||
|
||||
Each child track executes:
|
||||
|
||||
| Phase | Tasks | Output |
|
||||
|---|---|---|
|
||||
| **1. Acquire** | Run `extract_transcript.py` (always succeeds, fast). Run `download_video.py` if frame extraction needs video. | `transcript.json`, `download.log` |
|
||||
| **2. Keyframes** | Run `extract_keyframes.py` with sensible defaults (threshold 0.4). Manual review of frame set. | `frames/*.jpg`, `extraction_meta.json` |
|
||||
| **3. OCR** | Run `ocr_frames.py` on frames. Spot-check OCR quality. | `ocr.md` |
|
||||
| **4. Synthesis** | Tier 3 worker prompt: transcript + OCR + frame images → report.md (target 1000-10000 LOC). Human review + iteration. | `report.md`, `summary.md` |
|
||||
| **5. Verification** | Idempotency check (re-run scripts — should not break). Audit checklist. End-of-track report. | `tests/artifacts/<slug>/` |
|
||||
|
||||
### FR6. Per-video report structure (8 sections, target 1000-10000 LOC)
|
||||
|
||||
Each `report.md` follows this structure (mirrors `nagent_review`/`fable_review` style):
|
||||
|
||||
```
|
||||
# <Video Title>
|
||||
**Source:** <YouTube URL>
|
||||
**Author:** <Author>
|
||||
**Date Added to Campaign:** 2026-06-21
|
||||
**Cluster:** <A | B | C | D | E>
|
||||
**Slug:** <slug>
|
||||
|
||||
## 1. TL;DR (3-5 sentences)
|
||||
## 2. Key Concepts (5-15 bullets, each with brief explanation)
|
||||
## 3. Frame Analysis (one subsection per significant frame; embed image; describe visual content + OCR text + significance)
|
||||
## 4. Transcript Highlights (with timestamps; verbatim quotes of key passages)
|
||||
## 5. Mathematical / Theoretical Content (formal notation; derivations; references)
|
||||
## 6. Connections to Other Videos in Campaign (forward + backward links)
|
||||
## 7. Open Questions / Follow-up (what this video raises but doesn't answer)
|
||||
## 8. References (people, papers, prior work cited in the video)
|
||||
```
|
||||
|
||||
Plus a `summary.md` per video (200-400 words — quick reference for cross-cutting synthesis).
|
||||
|
||||
### FR7. Cross-cutting synthesis structure
|
||||
|
||||
The synthesis track produces:
|
||||
- `per_video_summary.md` — one paragraph (150-250 words) per video, the "summary of each video" the user requested. Ordered by execution order (matches umbrella §6).
|
||||
- `report.md` — the "summary report of key takeaways":
|
||||
1. **Theme matrix** across clusters A/B/C/D/E (which videos cover which themes)
|
||||
2. **Cross-video concept map** (which video introduced which idea; which video references which)
|
||||
3. **5-10 high-level takeaways** (the "what I learned that I didn't know before" section)
|
||||
4. **Mathematical prerequisite graph** (what math is needed to understand what)
|
||||
5. **Open research questions** (where the field is uncertain or contested)
|
||||
6. **Recommended next-watch list** (videos the user might want to find based on what they liked here)
|
||||
|
||||
### FR8. Storage & naming
|
||||
|
||||
- **mp4 files:** NEVER committed to git. Gitignored via pattern matching (per AGENTS.md file size conventions).
|
||||
- **Frame images:** committed if <500KB each; otherwise gitignored with `extraction_meta.json` (frame paths + hashes) committed.
|
||||
- **Transcripts, OCR, summaries, reports:** committed (small text files).
|
||||
- **Test artifacts:** `tests/artifacts/<slug>/` per AGENTS.md artifact isolation convention.
|
||||
|
||||
### FR9. Dependency graph
|
||||
|
||||
```
|
||||
UMBRELLA (video_analysis_campaign_20260621)
|
||||
├── child 1: video_analysis_cs229_building_llms_20260621
|
||||
├── child 2: video_analysis_probability_logic_20260621
|
||||
├── ...
|
||||
├── child 12: video_analysis_creikey_dl_cv_20260621
|
||||
└── SYNTHESIS: video_analysis_synthesis_20260621 (blocked_by all 12 children)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Non-Functional Requirements
|
||||
|
||||
- **Lossless preservation:** all artifacts stored in machine-readable form (JSON, plain text). Pass 2's de-obfuscation pass must be able to consume every artifact programmatically.
|
||||
- **TDD:** every new script in `scripts/video_analysis/` has tests in `tests/test_video_analysis_*.py` written BEFORE implementation (red phase first).
|
||||
- **Code style:** 1-space indent, type hints, no comments per `conductor/code_styleguides/python.md`. Result[T] error handling per `conductor/code_styleguides/error_handling.md`.
|
||||
- **No new pyproject.toml deps:** all tools (`yt-dlp`, `ffmpeg`, `cv2`, `imagehash`, `PIL`, `winsdk`/`tesseract`) are either system binaries (subprocess) or already in the project's venv (verify before installing).
|
||||
- **No `src/` changes:** research-only campaign. No modifications to `src/*.py`, no new `src/<thing>.py` files (per AGENTS.md File Size and Naming Convention rule).
|
||||
- **Per-task atomic commits:** each child track follows `conductor/workflow.md` per-task commit discipline.
|
||||
- **Git notes:** each task gets a git note summarizing what was done and why.
|
||||
- **No day estimates:** scope measured in files/sites per `conductor/workflow.md` Tier 1 Track Initialization Rules.
|
||||
|
||||
---
|
||||
|
||||
## 6. The 12 Videos in Execution Order
|
||||
|
||||
The order is: Stanford CS229 first (canonical ML foundation) → math foundations (A) → Platonic AI (B) → biological/cognitive (C, meta-first then concrete) → CS336 deep dive on architectures → applied capstone (D).
|
||||
|
||||
| # | Slug | Title | Cluster | YouTube ID |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `cs229_building_llms` | CS229 — Building LLMs | E | `9vM4p9NN0Ts` |
|
||||
| 2 | `probability_logic` | Probability Theory is an Extension of Logic | A | `0yF9TvMeAzM` |
|
||||
| 3 | `entropy_epiplexity` | From Entropy to Epiplexity (Wilson & Finzi) | A | `_U8AwUq_aJQ` |
|
||||
| 4 | `score_dynamics_giorgini` | Learning Dynamics from Statistics (Giorgini) | A | `P75iVMmbqQk` |
|
||||
| 5 | `platonic_intelligence_kumar` | Towards a Platonic Intelligence (Kumar) | B | `1mXUFweWOug` |
|
||||
| 6 | `free_lunches_levin` | Free Lunches (Levin) | B | `K8BmMU1Tm-I` |
|
||||
| 7 | `generic_systems_fields` | Interesting Behavior by Generic Systems (Fields) | C | `QeMajYvhEbI` |
|
||||
| 8 | `brain_counterintuitive` | Most Counterintuitive Way to Build a Brain | C | `cDxtFtoQVNc` |
|
||||
| 9 | `neural_dynamics_miller` | Cognition Emerges from Neural Dynamics (Miller) | C | `0BS-BzEFTXA` |
|
||||
| 10 | `multiscale_hoffman` | Multiscale Logic of Collective Intelligence (Hoffman & Prakash) | C | `YnfaT5APPB0` |
|
||||
| 11 | `cs336_architectures` | CS336 Lecture 3: Architectures | E | `lVynu4bo1rY` |
|
||||
| 12 | `creikey_dl_cv` | Creikey — DL/CV for Game Developers | D | `yxkUvXs-hoQ` |
|
||||
|
||||
---
|
||||
|
||||
## 7. Slug-to-URL Mapping
|
||||
|
||||
The full URL for each video (for reference; the child spec.md files reproduce these):
|
||||
|
||||
| Slug | URL |
|
||||
|---|---|
|
||||
| `cs229_building_llms` | `https://youtu.be/9vM4p9NN0Ts` |
|
||||
| `probability_logic` | `https://youtu.be/0yF9TvMeAzM` |
|
||||
| `entropy_epiplexity` | `https://youtu.be/_U8AwUq_aJQ` |
|
||||
| `score_dynamics_giorgini` | `https://youtu.be/P75iVMmbqQk` |
|
||||
| `platonic_intelligence_kumar` | `https://youtu.be/1mXUFweWOug` |
|
||||
| `free_lunches_levin` | `https://youtu.be/K8BmMU1Tm-I` |
|
||||
| `generic_systems_fields` | `https://youtu.be/QeMajYvhEbI` |
|
||||
| `brain_counterintuitive` | `https://youtu.be/cDxtFtoQVNc` |
|
||||
| `neural_dynamics_miller` | `https://youtu.be/0BS-BzEFTXA` |
|
||||
| `multiscale_hoffman` | `https://youtu.be/YnfaT5APPB0` |
|
||||
| `cs336_architectures` | `https://youtu.be/lVynu4bo1rY` |
|
||||
| `creikey_dl_cv` | `https://youtu.be/yxkUvXs-hoQ` |
|
||||
|
||||
---
|
||||
|
||||
## 8. Per-Video Report Structure (re-stated for emphasis)
|
||||
|
||||
The deep-dive report is the primary deliverable per child track. **Target: 1000-10000 LOC markdown per video** (per user directive 2026-06-21).
|
||||
|
||||
The 8-section structure from FR6 is MANDATORY. Each section gets roughly equal weight, but Frame Analysis + Math/Theoretical Content will likely dominate for technical videos. The Connections section is cross-referential — the synthesis track consumes it heavily.
|
||||
|
||||
---
|
||||
|
||||
## 9. Architecture Reference
|
||||
|
||||
This track does not modify the manual_slop application architecture. It produces research artifacts. The architecture refs that DO apply:
|
||||
|
||||
- **Track convention:** `conductor/workflow.md` "Standard Task Workflow" + "Tier 1 Track Initialization Rules" + per-task commit discipline
|
||||
- **Code style (for new scripts):** `conductor/code_styleguides/python.md` + `conductor/code_styleguides/error_handling.md`
|
||||
- **Artifact isolation:** AGENTS.md "test artifacts" → `tests/artifacts/<slug>/`
|
||||
- **Naming:** AGENTS.md "File Size and Naming Convention" → scripts in `scripts/<namespace>/`, no new `src/<thing>.py` files
|
||||
- **Multi-pass documentation:** this spec §11 explicitly records Pass 2/3 dependencies
|
||||
|
||||
---
|
||||
|
||||
## 10. Out of Scope (explicit)
|
||||
|
||||
- **Video analysis GUI panel in manual_slop** — no `src/gui_2.py` changes
|
||||
- **Auto-fetching of new videos on a schedule** — manual campaign execution only
|
||||
- **Building a video knowledge base** — separate from this analysis (per `conductor/code_styleguides/knowledge_artifacts.md`)
|
||||
- **The user's math encoding notation design itself** — Pass 2, USER-led, future track
|
||||
- **The projection-to-applied-domain methodology** — Pass 3, USER-led, future track
|
||||
- **Re-encoding or post-processing video files** — raw mp4s are downloaded once, not modified
|
||||
- **Auto-discovery of related videos on YouTube** — manual curation only
|
||||
- **Modifying any `src/*.py` files in manual_slop** — research-only campaign
|
||||
- **Adding `yt_dlp`, `cv2`, `imagehash`, `winsdk`, etc. to pyproject.toml** — all invoked via subprocess or via existing venv deps
|
||||
|
||||
---
|
||||
|
||||
## 11. Coordination with Future Passes (load-bearing)
|
||||
|
||||
### 11.1 Pass 2 (de-obfuscation via user's math encoding notation) — handoff contract
|
||||
|
||||
**Pass 2 will consume:**
|
||||
- `transcript.json` (every child track's `artifacts/transcript.json`)
|
||||
- `frames/*.jpg` (every child track's `artifacts/frames/`)
|
||||
- `ocr.md` (every child track's `artifacts/ocr.md`)
|
||||
- `report.md` (every child track's deep-dive report)
|
||||
- `summary.md` (every child track's summary)
|
||||
|
||||
**Pass 2's input encoding (user action item — pre-Pass-2):**
|
||||
- The user must rediscover/redefine their "compress/decompress math info" encoding notation.
|
||||
- This may be referenced in `conductor/tracks/intent_dsl_survey_20260612/` and other DSL-related track work; the user has prior art but it needs to be located.
|
||||
- Without this encoding system, Pass 2 cannot start.
|
||||
|
||||
**Pass 2 output:** a `deobfuscated/<slug>.md` per video + a `deobfuscated/synthesis.md` cross-cutting.
|
||||
|
||||
### 11.2 Pass 3 (projection to applied domain) — handoff contract
|
||||
|
||||
**Pass 3 will consume:** all of Pass 2's output + the user's stylistic preferences.
|
||||
|
||||
**Pass 3's input (user action item — pre-Pass-3):**
|
||||
- The user's stylistic preferences are documented in `conductor/workflow.md` (data-oriented design styleguide) and in the user's references to:
|
||||
- **Timothy Lottes** — GPGPU rendering, x56-40 / source-less programming (`C:\projects\forth\bootslop\references\`)
|
||||
- **Onat Türkçüoğlu** — Forth/ColorForth/VAMP/KYRA register-stack architecture (`C:\projects\forth\bootslop\`)
|
||||
- **Jebrim** — GPGPU community (specific reference TBD by user)
|
||||
- The user's "own caveats" are not yet documented — user must articulate these before Pass 3 starts.
|
||||
|
||||
**Pass 3 output:** applied-domain projections (e.g., "how would Lottes-style GPGPU kernels apply to inference?", "how would Onat's register-stack model apply to transformer attention?") + a synthesis.
|
||||
|
||||
### 11.3 Why this campaign is multi-pass
|
||||
|
||||
The user's framing (2026-06-21): "this large body of work encapsulated in the AI field which is largely impenetrable to me and associates." Pass 1 is information extraction + distillation (this track); Pass 2 is de-obfuscation (apply user's notation to make the math understandable); Pass 3 is projection (apply to user's domain). Each pass depends on the previous one's artifacts.
|
||||
|
||||
**Critical:** Pass 1 artifacts MUST be lossless. Over-summarization here is data loss that cascades.
|
||||
|
||||
---
|
||||
|
||||
## 12. Verification Criteria
|
||||
|
||||
The campaign is "done" when:
|
||||
|
||||
1. All 12 child tracks shipped (each with `report.md`, `summary.md`, `transcript.json`, `ocr.md`, frames extracted)
|
||||
2. Synthesis track shipped (with `per_video_summary.md` + `report.md`)
|
||||
3. All 5 scripts in `scripts/video_analysis/` shipped with passing tests
|
||||
4. Umbrella `README.md` lists all children with final status
|
||||
5. Campaign end-of-track report at `docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md`
|
||||
|
||||
The campaign is "Pass 1 complete" when:
|
||||
|
||||
- 12 + 1 = 13 child/synthesis tracks shipped
|
||||
- All artifacts preserved losslessly (verifiable by re-running scripts)
|
||||
- README.md shows all green
|
||||
|
||||
---
|
||||
|
||||
## 13. Risk Register
|
||||
|
||||
| ID | Title | Likelihood | Scope impact | Mitigation |
|
||||
|---|---|---|---|---|
|
||||
| R1 | `yt-dlp` not installed locally | High (verified at 2026-06-21: `yt-dlp` is NOT on PATH and NOT in this repo's venv) | First child track blocked until installed | Install `yt-dlp` via `pip install yt-dlp` in the repo's venv (single one-time task at the start of the first child track's execution) |
|
||||
| R2 | OCR quality insufficient for technical content | Medium | Some frames may have illegible text | Spot-check OCR per frame; manually transcribe critical frames in the report.md section |
|
||||
| R3 | Report exceeds 10000 LOC target | Low | User may want to split | Pass 2 can split; Pass 1 should not artificially cap |
|
||||
| R4 | Video mp4 files exceed disk space | Medium | Could hit quota | Delete mp4 after frame extraction (extract_frames.py already does this in bootslop) |
|
||||
| R5 | Two videos failed oEmbed fetch (private/age-restricted) | Confirmed for `9vM4p9NN0Ts` and `lVynu4bo1rY` | Unknown until track execution | User confirmed: `9vM4p9NN0Ts` = CS229, `lVynu4bo1rY` = CS336. The actual video data may still be accessible via `yt-dlp` (different from oEmbed) — verify in Phase 1 of each track |
|
||||
| R6 | User's math encoding notation (Pass 2) lost | Medium | Blocks Pass 2 | User action item: rediscover/redefine encoding before Pass 2 starts |
|
||||
| R7 | Pass 1 over-summarization loses signal for Pass 2 | Medium (if not enforced) | Cascades to Pass 2/3 | The "1000-10000 LOC target" + this spec's §0 explicit warning + per-section completeness check in verification |
|
||||
| R8 | Tier 2 capacity for 12+ child tracks | Medium | Tracks ship in sequence | Each child is independently shippable; the campaign is async |
|
||||
| R9 | Transcript API rate-limiting | Low | Some videos may fail on first fetch | Retry with backoff in `extract_transcript.py` |
|
||||
| R10 | `cv2` / `imagehash` not in this repo's venv | High (verified — they exist only in foreign venvs) | Blocks keyframe extraction | Install via `pip install opencv-python imagehash pillow` in the repo's venv (single one-time task) |
|
||||
|
||||
---
|
||||
|
||||
## 14. User Directives (recorded for next agent / future-self)
|
||||
|
||||
- **2026-06-21:** "Sure" — confirmed the 12-video order in §6.
|
||||
- **2026-06-21:** "This looks good, I'd say 2 [the report target]. should minimum 1000 and tops at 10k lines of markdown." — 1000-10000 LOC target per video report.
|
||||
- **2026-06-21:** "I want to add a note about this campaign, this is a first pass in a series of passes where we are doing essentially information extraction and distillation." — multi-pass framing; Pass 1 = this track.
|
||||
- **2026-06-21:** "Some of my preferences are within the workflow for conductor and are influenced by the 'handmade/data-oriented/GPGPU (Timothy Lottes, Onatt, Jebrim)' community along with my own caveats." — Pass 3 inputs.
|
||||
- **2026-06-21:** "These future passes after this first pass will be important to clarifying to my mind this large body of work encapsulated in the ai field which is largely impenetrable to me and associates." — campaign motivation.
|
||||
|
||||
---
|
||||
|
||||
## 15. See Also
|
||||
|
||||
- `conductor/workflow.md` — track convention, per-task commits, git notes, verification protocol
|
||||
- `conductor/code_styleguides/python.md` — 1-space indent, type hints, no comments
|
||||
- `conductor/code_styleguides/error_handling.md` — Result[T] pattern for new scripts
|
||||
- `conductor/code_styleguides/data_oriented_design.md` — referenced by Pass 3 (out of scope here)
|
||||
- `conductor/code_styleguides/agent_memory_dimensions.md` — referenced by Pass 2/3 for memory-shape decisions
|
||||
- `conductor/code_styleguides/knowledge_artifacts.md` — referenced by Pass 3 for knowledge-base shape
|
||||
- `conductor/tracks/intent_dsl_survey_20260612/` — prior DSL work that Pass 2 may build on
|
||||
- `conductor/tracks/nagent_review_20260608/report.md` — precedent for deep-dive report format
|
||||
- `conductor/tracks/fable_review_20260617/report.md` — precedent for synthesis report format
|
||||
- `C:\projects\forth\bootslop\download_videos.py`, `extract_frames.py`, `process_visuals.py` — reference scripts (NOT imported; new scripts in this repo's namespace)
|
||||
- `https://pypi.org/project/youtube-transcript-api/` — transcript extraction
|
||||
- `C:\projects\kasa\venv\Lib\site-packages\cv2\` — proves `cv2`/`ffmpeg` is installable in a Python venv on this machine
|
||||
@@ -0,0 +1,86 @@
|
||||
# Track state for video_analysis_campaign_20260621
|
||||
# Updated by Tier 1 Orchestrator (initially) and Tier 2 Tech Lead (during execution)
|
||||
|
||||
[meta]
|
||||
track_id = "video_analysis_campaign_20260621"
|
||||
name = "Video Analysis Campaign (12 videos, 5 clusters, 3 passes)"
|
||||
status = "active"
|
||||
current_phase = 0 # Phase 0 = tooling prerequisites (not yet started)
|
||||
last_updated = "2026-06-21"
|
||||
|
||||
[blocked_by]
|
||||
# Independent umbrella. No blockers.
|
||||
|
||||
[blocks]
|
||||
# This umbrella blocks the synthesis track:
|
||||
video_analysis_synthesis_20260621 = "planned"
|
||||
# Each child track is "blocked_by" the umbrella + its own dependencies (none)
|
||||
|
||||
[phases]
|
||||
phase_0 = { status = "pending", checkpointsha = "", name = "Tooling Prerequisites (yt-dlp, cv2, imagehash, OCR backend)" }
|
||||
phase_1 = { status = "pending", checkpointsha = "", name = "Reusable Tooling (5 scripts with TDD)" }
|
||||
phase_2 = { status = "pending", checkpointsha = "", name = "Per-Child Tracks (12 videos, 5-phase pipeline each)" }
|
||||
phase_3 = { status = "pending", checkpointsha = "", name = "Synthesis Track (blocked by all 12 children)" }
|
||||
phase_4 = { status = "pending", checkpointsha = "", name = "Campaign Closeout (README update, end-of-track report, archive, chronology)" }
|
||||
|
||||
[tasks]
|
||||
# Phase 0 tasks
|
||||
t0_1 = { status = "pending", commit_sha = "", description = "Install yt-dlp in this repo's venv (pip install yt-dlp). Verify with python -c \"import yt_dlp; print(yt_dlp.version.__version__)\"." }
|
||||
t0_2 = { status = "pending", commit_sha = "", description = "Install opencv-python, imagehash, pillow in this repo's venv. Verify imports." }
|
||||
t0_3 = { status = "pending", commit_sha = "", description = "Decide on OCR backend: try winsdk first (matches bootslop), fall back to tesseract if winsdk proves problematic." }
|
||||
t0_4 = { status = "pending", commit_sha = "", description = "Create scripts/video_analysis/ namespace and tests/test_video_analysis_*.py skeleton (empty placeholder files)." }
|
||||
|
||||
# Phase 1 tasks (script TDD)
|
||||
t1_1 = { status = "pending", commit_sha = "", description = "Write tests for extract_transcript.py (red phase): success path, network error, missing video ID, malformed JSON response, retry behavior." }
|
||||
t1_2 = { status = "pending", commit_sha = "", description = "Implement extract_transcript.py (green phase). CLI: --url, --output, --json. Outputs transcript.json with segments + plain + metadata." }
|
||||
t1_3 = { status = "pending", commit_sha = "", description = "Write tests for download_video.py (red)." }
|
||||
t1_4 = { status = "pending", commit_sha = "", description = "Implement download_video.py (green). Subprocess yt-dlp. Outputs mp4 + download.log." }
|
||||
t1_5 = { status = "pending", commit_sha = "", description = "Write tests for extract_keyframes.py (red)." }
|
||||
t1_6 = { status = "pending", commit_sha = "", description = "Implement extract_keyframes.py (green). ffmpeg scene detect + cv2 + imagehash dedup. Outputs frames/*.jpg + extraction_meta.json." }
|
||||
t1_7 = { status = "pending", commit_sha = "", description = "Write tests for ocr_frames.py (red)." }
|
||||
t1_8 = { status = "pending", commit_sha = "", description = "Implement ocr_frames.py (green). Winsdk (or tesseract fallback). Outputs ocr.md." }
|
||||
t1_9 = { status = "pending", commit_sha = "", description = "Write tests for synthesize_report.py (red)." }
|
||||
t1_10 = { status = "pending", commit_sha = "", description = "Implement synthesize_report.py (green). Orchestrator. Outputs artifacts/ + report.md stub + summary.md stub." }
|
||||
|
||||
# Phase 2 tasks (12 child tracks - one entry each; details in child plans)
|
||||
t2_1 = { status = "pending", commit_sha = "", description = "Child 1: video_analysis_cs229_building_llms_20260621 - verify yt-dlp access (oEmbed failed 401), execute 5-phase pipeline, ship report.md (1000-10000 LOC) + summary.md" }
|
||||
t2_2 = { status = "pending", commit_sha = "", description = "Child 2: video_analysis_probability_logic_20260621 - execute 5-phase pipeline" }
|
||||
t2_3 = { status = "pending", commit_sha = "", description = "Child 3: video_analysis_entropy_epiplexity_20260621 - execute 5-phase pipeline" }
|
||||
t2_4 = { status = "pending", commit_sha = "", description = "Child 4: video_analysis_score_dynamics_giorgini_20260621 - execute 5-phase pipeline" }
|
||||
t2_5 = { status = "pending", commit_sha = "", description = "Child 5: video_analysis_platonic_intelligence_kumar_20260621 - execute 5-phase pipeline" }
|
||||
t2_6 = { status = "pending", commit_sha = "", description = "Child 6: video_analysis_free_lunches_levin_20260621 - execute 5-phase pipeline" }
|
||||
t2_7 = { status = "pending", commit_sha = "", description = "Child 7: video_analysis_generic_systems_fields_20260621 - execute 5-phase pipeline" }
|
||||
t2_8 = { status = "pending", commit_sha = "", description = "Child 8: video_analysis_brain_counterintuitive_20260621 - execute 5-phase pipeline" }
|
||||
t2_9 = { status = "pending", commit_sha = "", description = "Child 9: video_analysis_neural_dynamics_miller_20260621 - execute 5-phase pipeline" }
|
||||
t2_10 = { status = "pending", commit_sha = "", description = "Child 10: video_analysis_multiscale_hoffman_20260621 - execute 5-phase pipeline" }
|
||||
t2_11 = { status = "pending", commit_sha = "", description = "Child 11: video_analysis_cs336_architectures_20260621 - verify yt-dlp access (oEmbed failed 401), execute 5-phase pipeline" }
|
||||
t2_12 = { status = "pending", commit_sha = "", description = "Child 12: video_analysis_creikey_dl_cv_20260621 - execute 5-phase pipeline" }
|
||||
|
||||
# Phase 3 tasks (synthesis)
|
||||
t3_1 = { status = "pending", commit_sha = "", description = "Initialize video_analysis_synthesis_20260621 (spec.md + plan.md + metadata.json + state.toml)" }
|
||||
t3_2 = { status = "pending", commit_sha = "", description = "Execute synthesis: consume 12 children's report.md + summary.md, produce per_video_summary.md + report.md" }
|
||||
|
||||
# Phase 4 tasks (closeout)
|
||||
t4_1 = { status = "pending", commit_sha = "", description = "Update umbrella README.md with final statuses (all 12 children + synthesis shipped)" }
|
||||
t4_2 = { status = "pending", commit_sha = "", description = "Write end-of-track report at docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md" }
|
||||
t4_3 = { status = "pending", commit_sha = "", description = "Move umbrella + 13 children to conductor/archive/ per project convention" }
|
||||
t4_4 = { status = "pending", commit_sha = "", description = "Update conductor/chronology.md with 14 new rows" }
|
||||
|
||||
[verification]
|
||||
# These flip to true as the campaign progresses
|
||||
tooling_installed = false
|
||||
scripts_tdd_complete = false
|
||||
all_12_children_shipped = false
|
||||
synthesis_shipped = false
|
||||
end_of_track_report_committed = false
|
||||
future_pass_hooks_intact = false
|
||||
|
||||
[user_directives_logged]
|
||||
order_confirmed = "Per user 2026-06-21: 12-video sequence in spec.md §6"
|
||||
report_loc_target = "Per user 2026-06-21: minimum 1000 LOC, maximum 10000 LOC markdown per video report"
|
||||
multi_pass_framing = "Per user 2026-06-21: Pass 1 = information extraction (this track), Pass 2 = de-obfuscation, Pass 3 = projection"
|
||||
lossless_preservation = "Per user 2026-06-21: Pass 1 artifacts must be lossless; over-summarization is data loss for Pass 2"
|
||||
stanford_mapping = "Per user 2026-06-21: 9vM4p9NN0Ts = CS229 (Building LLMs), lVynu4bo1rY = CS336 Lecture 3 (Architectures)"
|
||||
campaign_motivation = "Per user 2026-06-21: 'large body of work encapsulated in the ai field which is largely impenetrable to me and associates'"
|
||||
stylistic_influences = "Per user 2026-06-21: handmade/data-oriented/GPGPU (Timothy Lottes, Onat Türkçüoğlu, Jebrim) + user's own caveats - referenced for Pass 3"
|
||||
no_day_estimates = "Per conductor/workflow.md Tier 1 Track Initialization Rules (added 2026-06-16). Scope measured in files/sites only."
|
||||
Reference in New Issue
Block a user