ed/manual_slop
Private
Public Access
0
0
Fork 0
Code Activity
Files
46a2245658438e9388bbcce42cf3f3504c4cea0a
manual_slop/conductor/tracks/video_analysis_campaign_20260621/plan.md
T

9.3 KiB
Raw Blame History

Plan: Video Analysis Campaign — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build the reusable tooling (5 scripts in scripts/video_analysis/) + execute the 5-phase pipeline for each of 12 curated YouTube videos, producing per-video deep-dive reports + a cross-cutting synthesis.

Architecture: Campaign-level umbrella + 12 child tracks + 1 synthesis track (14 folders total). Phase 0 installs tooling; Phase 1 builds the reusable scripts (TDD); Phase 2 runs the pipeline per video (each child is a track); Phase 3 synthesizes; Phase 4 closes out. Lossless preservation directive (1000-10000 LOC per video report).

Tech Stack:

  • Python 3.11+ (1-space indent, type hints, no comments per conductor/code_styleguides/python.md)
  • Result[T] error handling per conductor/code_styleguides/error_handling.md
  • yt-dlp (subprocess), youtube-transcript-api (import), ffmpeg + cv2 + imagehash + PIL (subprocess + import), winsdk or tesseract (OCR)
  • uv run pytest for tests (per conductor/code_styleguides/workspace_paths.md)

Reference scripts (DO NOT import — reference only):

  • C:/projects/forth/bootslop/download_videos.py — yt-dlp usage
  • C:/projects/forth/bootslop/extract_frames.py — cv2 + imagehash
  • C:/projects/forth/bootslop/process_visuals.py — winsdk OCR + visual heuristics
  • C:/projects/forth/bootslop/ocr_interaction.py — standalone OCR

Phase 0: Tooling Prerequisites

One-time setup. Must complete before any script work begins.

Task 0.1: Install yt-dlp

Files: none modified. pyproject.toml deps updated (manually).

  • Step 1: Install yt-dlp in the repo's venv — DONE (uv pip install yt-dlp → yt-dlp 2026.06.09)

  • Step 2: Verify import — DONE (import yt_dlp works)

  • Step 3: Verify CLI availability (for subprocess invocation) — DONE (uv run yt-dlp --version → 2026.06.09)

  • Step 4: Commit — DONE (commit 1c05305a)

Task 0.2: Install opencv-python, imagehash, pillow

Files: none modified. pyproject.toml deps updated.

  • Step 1: Install packages — DONE (uv pip install opencv-python imagehash pillow → 4.10.0 / 4.3.2 / 11.0.0)

  • Step 2: Verify imports — DONE

  • Step 3: Verify ffmpeg is on PATH (cv2 needs it) — DONE (ffmpeg 8.1.1 confirmed)

  • Step 4: Commit — DONE (combined with Task 0.1 into 1c05305a)

Task 0.3: OCR backend decision (winsdk vs tesseract)

Files: none modified. Decision recorded in spec.md §4 / metadata.json.

  • Step 1: Try installing winsdk first (matches bootslop pattern) — DONE (winsdk 1.0.0b10)

  • Step 2: Verify winsdk can be imported and OCR engine can be created — DONE (engine available for en-US)

  • Step 3: If winsdk fails, fall back to tesseract — N/A (winsdk works); pytesseract installed as fallback

  • Step 4: Record decision in spec.md §4 (OCR backend line) — winsdk default

  • Step 5: Commit decision — DONE (combined into 1c05305a)

Task 0.4: Create scripts/video_analysis/ namespace + tests skeleton

Files: Create scripts/video_analysis/__init__.py, tests/test_video_analysis_placeholder.py.

  • Step 1: Create scripts/video_analysis/ directory — DONE

  • Step 2: Create __init__.py — DONE

  • Step 3: Create placeholder test file — DONE

  • Step 4: Verify placeholder test passes — DONE (1 passed; later replaced in Task 1.1)

  • Step 5: Commit — DONE (commit 12fcc55c)

Phase 1: Reusable Tooling (5 scripts, TDD)

Each script is independently TDD-tested. Order: extract_transcript → download_video → extract_keyframes → ocr_frames → synthesize_report (synthesize_report is last because it composes the others).

Common conventions (apply to all 5 scripts):

  • 1-space indent, type hints on all params/returns, no comments in implementation code
  • Use Result[T, ErrorInfo] per conductor/code_styleguides/error_handling.md
  • CLI interface: --url/--video/--frames-dir input, --output/--output-dir target, --json machine-readable mode, --help
  • subprocess.run for yt-dlp / ffmpeg / tesseract; never import the binaries

Task 1.1: Write tests + implement extract_transcript.py

  • Step 1-7: TDD complete — DONE (commit 94f4a4ee). 8 tests passing.

Task 1.2: Write tests + implement download_video.py

  • Step 1-5: TDD complete — DONE (commit 45a5e814). 5 tests passing.

Task 1.3: Write tests + implement extract_keyframes.py

  • Step 1-5: TDD complete — DONE (commit 9ccdedee). 4 tests passing.

Task 1.4: Write tests + implement ocr_frames.py

  • Step 1-5: TDD complete — DONE (commit ed0d198a). 4 tests passing.

Task 1.5: Write tests + implement synthesize_report.py

  • Step 1-5: TDD complete — DONE (commit 548c4fef). 5 tests passing.

Phase 2: Per-Child Tracks (12 tracks, brief)

Each child track has its own conductor/tracks/video_analysis_<slug>_20260621/plan.md (scaffolded by umbrella Tier 2 in init commit c1a15c45). The umbrella plan does NOT enumerate per-child tasks — each child is independently shippable.

5-phase pipeline (per child):

  1. Acquire — Run extract_transcript.py + download_video.py. Phase 1 deliverable for E-cluster children: verify yt-dlp access.
  2. Keyframes — Run extract_keyframes.py with threshold 0.4. Manual review of frame set.
  3. OCR — Run ocr_frames.py. Spot-check.
  4. Synthesis — Tier 3 worker writes report.md (1000-10000 LOC) + summary.md (200-400 words). Human review + iteration.
  5. Verification — Idempotency check, audit checklist, end-of-track report.

Execution order: per umbrella spec.md §6 (cs229 first, creikey last). Per-child execution is via separate Tier 2 dispatches.

Phase 3: Synthesis Track (brief)

After all 12 child tracks ship, the synthesis track consumes their report.md + summary.md and produces per_video_summary.md + report.md (per umbrella spec §FR7).

Synthesis track scaffolded in init commit c1a15c45. Execution is via separate Tier 2 dispatch after all children ship.

Phase 4: Campaign Closeout (brief)

  • Update umbrella README.md with final statuses.
  • Write end-of-track report at docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621.md.
  • Move umbrella + 13 children to conductor/archive/.
  • Update conductor/chronology.md with 14 new rows.

Verification (gate per workflow.md)

Each child track's Phase 5 must include:

  • Idempotency check (re-running scripts produces identical outputs modulo timestamps)
  • Audit checklist (every section of the report is populated, no "TBD")
  • All tests pass (uv run pytest tests/test_video_analysis_*.py)
  • Per-task commits with git notes

The campaign-level verification (Phase 4) confirms all 12 children + 1 synthesis are shipped and the future-pass hooks (umbrella spec §11) are intact.

Self-review

After writing this plan, the following checks were performed:

  1. Spec coverage: Every FR in umbrella spec.md §4 is mapped to a task here:

    • FR1 (umbrella + README): Task 0.4 + Phase 4 task 4.1
    • FR2 (12 child folders): each child's spec.md already exists; Phase 2 deferred to child plans
    • FR3 (1 synthesis folder): child spec.md exists; Phase 3 deferred
    • FR4 (5 scripts): Tasks 1.1-1.5
    • FR5 (5-phase pipeline): Phase 2 above
    • FR6 (8-section report): build_report_stub in Task 1.5 has all 8 sections
    • FR7 (6-section synthesis): synthesis spec.md (already written)
    • FR8 (storage): addressed in commits (gitignore for *.mp4 + large frames)
    • FR9 (dependency graph): addressed in umbrella spec + state.toml

  2. Placeholder scan: No "TBD", "TODO", "implement later" in the task steps. <Title TBD> appears in build_summary_stub as a placeholder that the Tier 3 worker fills in — this is intentional per FR5 phase 4 ("Tier 3 worker prompt: ... populate each section").

  3. Type consistency: All scripts use the same _Ok/_Err dataclass pattern (per the Result[T] convention in conductor/code_styleguides/error_handling.md). All scripts have a ok()/err() factory pair. The extract_transcript script imports _fetch_raw_transcript from within the same module — consistent across all 5 scripts.

Execution status (as of 2026-06-21)

Phase 0+1+2 init complete. 9 commits. 26 TDD tests passing.

Phase Status Notes
Phase 0 (Tooling) DONE 7 deps installed in repo venv. winsdk OCR verified.
Phase 1 (5 Scripts) DONE 26/26 tests passing. Result[T] convention applied.
Phase 2 init DONE 12 child + 1 synthesis tracks scaffolded.
Phase 2 execution (12 children) PENDING Per-child Tier 2 dispatches (12 invocations).
Phase 3 (synthesis) PENDING 1 Tier 2 dispatch after all children ship.
Phase 4 (closeout) PENDING Final umbrella dispatch after all 13 children + synthesis ship.

Interim end-of-track report: docs/reports/TRACK_COMPLETION_video_analysis_campaign_20260621_phase0_1_2init.md

Next steps: User dispatches Tier 2 per child via /tier-2-auto-execute video_analysis_<slug>_20260621 --resume. After all 12 + synthesis ship, user invokes final umbrella closeout (Phase 4).

View Git Blame Copy Permalink