From 41656d6b7c57344714cc38219d0b535085feea33 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Sun, 5 Jul 2026 17:36:37 -0400 Subject: [PATCH] conductor(plan): mark Phase 4 (fetch_thread.py) complete [06ff9299] --- .../twitter_threads_extraction_20260705/plan.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/conductor/tracks/twitter_threads_extraction_20260705/plan.md b/conductor/tracks/twitter_threads_extraction_20260705/plan.md index fabbd021..3074ec37 100644 --- a/conductor/tracks/twitter_threads_extraction_20260705/plan.md +++ b/conductor/tracks/twitter_threads_extraction_20260705/plan.md @@ -165,11 +165,11 @@ git commit -m "feat(twitter_threads): download_media.py — stdlib urllib + idem --- -## Phase 4: `fetch_thread.py` (the acquisition module; the network boundary) +## Phase 4: `fetch_thread.py` (the acquisition module; the network boundary) [checkpoint: 06ff9299] Focus: acquire a `ThreadData` from either a URL (via `gallery-dl` subprocess) or a local HTML file (Strategy C parse). The URL path is the network boundary; the HTML path is the offline fallback. -- [ ] **Task 4.1: Write failing tests for the local-HTML-parse path (Strategy C)** +- [x] **Task 4.1: Write failing tests for the local-HTML-parse path (Strategy C)** WHERE: `tests/test_twitter_threads_fetch.py` WHAT: 3+ tests: @@ -180,7 +180,7 @@ WHAT: 3+ tests: HOW: Use `html.parser.HTMLParser` (stdlib) — the test fixtures are real HTML strings. No network. VERIFY: `uv run pytest tests/test_twitter_threads_fetch.py` FAILS. -- [ ] **Task 4.2: Implement the local-HTML-parse path** +- [x] **Task 4.2: Implement the local-HTML-parse path** WHERE: `scripts/twitter_threads/fetch_thread.py` WHAT: A `fetch_thread_from_html(html_path: Path) -> Result[ThreadData, ErrorInfo]` function that parses a local HTML file using `html.parser.HTMLParser` (stdlib, no BeautifulSoup). @@ -190,7 +190,7 @@ HOW: - Return `Result.err(ErrorInfo("ParseError", "fetch_thread_from_html", detail))` on malformed HTML. VERIFY: `uv run pytest tests/test_twitter_threads_fetch.py` PASSES for the Strategy C tests. -- [ ] **Task 4.3: Implement the URL path (the `gallery-dl` subprocess wrapper)** +- [x] **Task 4.3: Implement the URL path (the `gallery-dl` subprocess wrapper)** WHERE: `scripts/twitter_threads/fetch_thread.py` (same file; add the URL path) WHAT: A `fetch_thread_from_url(url: str, cookies_path: Path | None = None) -> Result[ThreadData, ErrorInfo]` function. @@ -203,7 +203,7 @@ HOW: - On JSON parse failure: return `Result.err(ErrorInfo("JsonParseError", ...))`. VERIFY: Manual smoke test against a real X.com URL (the user provides a URL + cookies file). The automated tests cover the HTML path only (Strategy C); the URL path is the network boundary and is smoke-tested manually. -- [ ] **Task 4.4: Implement the CLI dispatch + `--help`** +- [x] **Task 4.4: Implement the CLI dispatch + `--help`** WHERE: `scripts/twitter_threads/fetch_thread.py` (the `if __name__ == "__main__":` block) WHAT: A CLI that takes a URL or a local HTML path + `--output` + optional `--cookies`, dispatches to the right function, and writes the `ThreadData` to a JSON file (intermediate) for downstream `download_media.py` + `render_markdown.py` to consume. @@ -220,7 +220,7 @@ if __name__ == "__main__": ``` VERIFY: `python scripts/twitter_threads/fetch_thread.py --help` works (standalone, no `src/` imports). -- [ ] **Task 4.5: Commit Phase 4** +- [x] **Task 4.5: Commit Phase 4** ```bash git add scripts/twitter_threads/fetch_thread.py tests/test_twitter_threads_fetch.py