Pass 1 of 3 user research campaign (12 videos, 5 clusters). - Umbrella: spec.md (full design), plan.md, metadata.json, state.toml, README.md - Multi-pass framing (Pass 2 de-obfuscation, Pass 3 projection) - Lossless preservation directive (1000-10000 LOC per video report target) - Tooling prerequisites: yt-dlp, cv2, imagehash install in repo venv - 5 reusable scripts to live in scripts/video_analysis/ (TDD) - 12 children + 1 synthesis = 14 folders total
4.9 KiB
Plan: Video Analysis Campaign (umbrella)
This is the umbrella-level plan. Per the Tier 1 Track Initialization Rules, scope is measured in files/sites — no day estimates.
Phase 0: Tooling Prerequisites
Must be completed before any child track can ship. One-time setup.
- Task 0.1: Install
yt-dlpin this repo's venv (pip install yt-dlp). Verify withpython -c "import yt_dlp; print(yt_dlp.version.__version__)". - Task 0.2: Install
opencv-python,imagehash,pillowin this repo's venv. Verify imports. - Task 0.3: Decide on OCR backend. Try
winsdkfirst (matches bootslop); fall back totesseractifwinsdkproves problematic. - Task 0.4: Create
scripts/video_analysis/namespace andtests/test_video_analysis_*.pyskeleton.
Phase 1: Reusable Tooling (5 scripts, TDD)
Each script is independently TDD-tested. Order: extract_transcript → download_video → extract_keyframes → ocr_frames → synthesize_report (synthesize_report is last because it composes the others).
- Task 1.1: Write tests for
extract_transcript.py(red). Tests cover: success path, network error, missing video ID, malformed JSON response, retry behavior. - Task 1.2: Implement
extract_transcript.py(green). CLI:--url,--output,--json. Outputstranscript.jsonwithsegments(list of{start, duration, text}) +plain(joined text) +metadata(video ID, fetch timestamp). - Task 1.3: Write tests for
download_video.py(red). - Task 1.4: Implement
download_video.py(green). CLI:--url,--output,--json. Subprocessyt-dlp. Outputsdownload.log. - Task 1.5: Write tests for
extract_keyframes.py(red). - Task 1.6: Implement
extract_keyframes.py(green). CLI:--video,--output-dir,--threshold,--json. Uses ffmpegselect=gt(scene\,0.4)+ cv2 frame extraction + imagehash dedup. Outputsframes/*.jpg+extraction_meta.json. - Task 1.7: Write tests for
ocr_frames.py(red). - Task 1.8: Implement
ocr_frames.py(green). CLI:--frames-dir,--output,--json. Uses winsdk (with tesseract fallback). Outputsocr.mdwith one section per frame (image path + OCR text + timestamp). - Task 1.9: Write tests for
synthesize_report.py(red). - Task 1.10: Implement
synthesize_report.py(green). CLI:--url,--slug,--output-dir,--json. Orchestrates the full pipeline for one video. Outputsartifacts/populated +report.mdstub +summary.mdstub.
Phase 2: Per-Child Tracks (12 tracks)
Each child track follows the 5-phase pipeline. The Tier 2 Tech Lead executes these. The umbrella plan does NOT enumerate per-child tasks — those live in each child's plan.md (created during execution).
Execution order (per §6 of spec.md):
| # | Slug | Cluster | Notes |
|---|---|---|---|
| 1 | cs229_building_llms |
E | Verify yt-dlp access (oEmbed failed 401) |
| 2 | probability_logic |
A | |
| 3 | entropy_epiplexity |
A | |
| 4 | score_dynamics_giorgini |
A | |
| 5 | platonic_intelligence_kumar |
B | |
| 6 | free_lunches_levin |
B | |
| 7 | generic_systems_fields |
C | |
| 8 | brain_counterintuitive |
C | |
| 9 | neural_dynamics_miller |
C | |
| 10 | multiscale_hoffman |
C | |
| 11 | cs336_architectures |
E | Verify yt-dlp access (oEmbed failed 401) |
| 12 | creikey_dl_cv |
D |
Note on E-cluster yt-dlp verification: the oEmbed API returned 401 for 9vM4p9NN0Ts and lVynu4bo1rY. This may be an oEmbed-specific restriction (private/age-restricted); yt-dlp may still work. Phase 1 of those child tracks must verify yt-dlp access before downloading anything.
Phase 3: Synthesis Track (blocked by all 12 children)
After all 12 child tracks ship, the synthesis track consumes their outputs and produces:
per_video_summary.md(the "summary of each video" the user requested)report.md(the "summary report of key takeaways" — theme matrix, concept map, top takeaways, math prereq graph, open questions, next-watch list)
Phase 4: Campaign Closeout
- Task 4.1: Update umbrella
README.mdwith final statuses (all 12 children + synthesis shipped). - Task 4.2: Write end-of-track report at
docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md. - Task 4.3: Move umbrella + 13 children to
conductor/archive/per the project's archiving convention. - Task 4.4: Update
conductor/chronology.mdwith the 14 track rows.
Verification (gate per workflow.md)
Each child track's Phase 5 must include:
- Idempotency check: re-running scripts produces identical outputs (modulo timestamps)
- Audit checklist: every section of the report is populated, no "TBD"
- Tests pass
- Per-task commits with git notes
The campaign-level verification (Phase 4.2 end-of-track report) confirms all 12 children + 1 synthesis are shipped and the future-pass hooks (§11 of spec.md) are intact.