ed/manual_slop
Private
Public Access
0
0
Fork 0
Code Activity

conductor(video_analysis_campaign): Initialize umbrella track + 12 child + 1 synthesis scaffold

Browse Source 
Pass 1 of 3 user research campaign (12 videos, 5 clusters).
- Umbrella: spec.md (full design), plan.md, metadata.json, state.toml, README.md
- Multi-pass framing (Pass 2 de-obfuscation, Pass 3 projection)
- Lossless preservation directive (1000-10000 LOC per video report target)
- Tooling prerequisites: yt-dlp, cv2, imagehash install in repo venv
- 5 reusable scripts to live in scripts/video_analysis/ (TDD)
- 12 children + 1 synthesis = 14 folders total
This commit is contained in:
ed
2026-06-21 15:02:44 -04:00
parent aca84b881b
commit b184250b78
5 changed files with 881 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conductor/tracks/video_analysis_campaign_20260621/plan.md
+73
View File
@@ -0,0 +1,73 @@
# Plan: Video Analysis Campaign (umbrella)
This is the umbrella-level plan. Per the Tier 1 Track Initialization Rules, scope is measured in files/sites — no day estimates.
## Phase 0: Tooling Prerequisites
Must be completed before any child track can ship. One-time setup.
- [ ] **Task 0.1:** Install `yt-dlp` in this repo's venv (`pip install yt-dlp`). Verify with `python -c "import yt_dlp; print(yt_dlp.version.__version__)"`.
- [ ] **Task 0.2:** Install `opencv-python`, `imagehash`, `pillow` in this repo's venv. Verify imports.
- [ ] **Task 0.3:** Decide on OCR backend. Try `winsdk` first (matches bootslop); fall back to `tesseract` if `winsdk` proves problematic.
- [ ] **Task 0.4:** Create `scripts/video_analysis/` namespace and `tests/test_video_analysis_*.py` skeleton.
## Phase 1: Reusable Tooling (5 scripts, TDD)
Each script is independently TDD-tested. Order: extract_transcript → download_video → extract_keyframes → ocr_frames → synthesize_report (synthesize_report is last because it composes the others).
- [ ] **Task 1.1:** Write tests for `extract_transcript.py` (red). Tests cover: success path, network error, missing video ID, malformed JSON response, retry behavior.
- [ ] **Task 1.2:** Implement `extract_transcript.py` (green). CLI: `--url`, `--output`, `--json`. Outputs `transcript.json` with `segments` (list of `{start, duration, text}`) + `plain` (joined text) + `metadata` (video ID, fetch timestamp).
- [ ] **Task 1.3:** Write tests for `download_video.py` (red).
- [ ] **Task 1.4:** Implement `download_video.py` (green). CLI: `--url`, `--output`, `--json`. Subprocess `yt-dlp`. Outputs `download.log`.
- [ ] **Task 1.5:** Write tests for `extract_keyframes.py` (red).
- [ ] **Task 1.6:** Implement `extract_keyframes.py` (green). CLI: `--video`, `--output-dir`, `--threshold`, `--json`. Uses ffmpeg `select=gt(scene\,0.4)` + cv2 frame extraction + imagehash dedup. Outputs `frames/*.jpg` + `extraction_meta.json`.
- [ ] **Task 1.7:** Write tests for `ocr_frames.py` (red).
- [ ] **Task 1.8:** Implement `ocr_frames.py` (green). CLI: `--frames-dir`, `--output`, `--json`. Uses winsdk (with tesseract fallback). Outputs `ocr.md` with one section per frame (image path + OCR text + timestamp).
- [ ] **Task 1.9:** Write tests for `synthesize_report.py` (red).
- [ ] **Task 1.10:** Implement `synthesize_report.py` (green). CLI: `--url`, `--slug`, `--output-dir`, `--json`. Orchestrates the full pipeline for one video. Outputs `artifacts/` populated + `report.md` stub + `summary.md` stub.
## Phase 2: Per-Child Tracks (12 tracks)
Each child track follows the 5-phase pipeline. The Tier 2 Tech Lead executes these. The umbrella plan does NOT enumerate per-child tasks — those live in each child's `plan.md` (created during execution).
**Execution order** (per §6 of spec.md):
| # | Slug | Cluster | Notes |
|---|------|---------|-------|
| 1 | `cs229_building_llms` | E | Verify yt-dlp access (oEmbed failed 401) |
| 2 | `probability_logic` | A | |
| 3 | `entropy_epiplexity` | A | |
| 4 | `score_dynamics_giorgini` | A | |
| 5 | `platonic_intelligence_kumar` | B | |
| 6 | `free_lunches_levin` | B | |
| 7 | `generic_systems_fields` | C | |
| 8 | `brain_counterintuitive` | C | |
| 9 | `neural_dynamics_miller` | C | |
| 10 | `multiscale_hoffman` | C | |
| 11 | `cs336_architectures` | E | Verify yt-dlp access (oEmbed failed 401) |
| 12 | `creikey_dl_cv` | D | |
**Note on E-cluster yt-dlp verification:** the oEmbed API returned 401 for `9vM4p9NN0Ts` and `lVynu4bo1rY`. This may be an oEmbed-specific restriction (private/age-restricted); `yt-dlp` may still work. Phase 1 of those child tracks must verify yt-dlp access before downloading anything.
## Phase 3: Synthesis Track (blocked by all 12 children)
After all 12 child tracks ship, the synthesis track consumes their outputs and produces:
- `per_video_summary.md` (the "summary of each video" the user requested)
- `report.md` (the "summary report of key takeaways" — theme matrix, concept map, top takeaways, math prereq graph, open questions, next-watch list)
## Phase 4: Campaign Closeout
- [ ] **Task 4.1:** Update umbrella `README.md` with final statuses (all 12 children + synthesis shipped).
- [ ] **Task 4.2:** Write end-of-track report at `docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md`.
- [ ] **Task 4.3:** Move umbrella + 13 children to `conductor/archive/` per the project's archiving convention.
- [ ] **Task 4.4:** Update `conductor/chronology.md` with the 14 track rows.
## Verification (gate per workflow.md)
Each child track's Phase 5 must include:
- [ ] Idempotency check: re-running scripts produces identical outputs (modulo timestamps)
- [ ] Audit checklist: every section of the report is populated, no "TBD"
- [ ] Tests pass
- [ ] Per-task commits with git notes
The campaign-level verification (Phase 4.2 end-of-track report) confirms all 12 children + 1 synthesis are shipped and the future-pass hooks (§11 of spec.md) are intact.