ed/manual_slop
Private
Public Access
0
0
Fork 0
Code Activity

conductor(plan): mark Phase 4 (fetch_thread.py) complete [06ff9299]

Browse Source
This commit is contained in:
ed
2026-07-05 17:36:37 -04:00
parent 06ff9299e2
commit 41656d6b7c
1 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conductor/tracks/twitter_threads_extraction_20260705/plan.md
+6 -6
View File
@@ -165,11 +165,11 @@ git commit -m "feat(twitter_threads): download_media.py — stdlib urllib + idem
---
## Phase 4: `fetch_thread.py` (the acquisition module; the network boundary)
## Phase 4: `fetch_thread.py` (the acquisition module; the network boundary) [checkpoint: 06ff9299]
Focus: acquire a `ThreadData` from either a URL (via `gallery-dl` subprocess) or a local HTML file (Strategy C parse). The URL path is the network boundary; the HTML path is the offline fallback.
- [ ] **Task 4.1: Write failing tests for the local-HTML-parse path (Strategy C)**
- [x] **Task 4.1: Write failing tests for the local-HTML-parse path (Strategy C)**
WHERE: `tests/test_twitter_threads_fetch.py`
WHAT: 3+ tests:
@@ -180,7 +180,7 @@ WHAT: 3+ tests:
HOW: Use `html.parser.HTMLParser` (stdlib) — the test fixtures are real HTML strings. No network.
VERIFY: `uv run pytest tests/test_twitter_threads_fetch.py` FAILS.
- [ ] **Task 4.2: Implement the local-HTML-parse path**
- [x] **Task 4.2: Implement the local-HTML-parse path**
WHERE: `scripts/twitter_threads/fetch_thread.py`
WHAT: A `fetch_thread_from_html(html_path: Path) -> Result[ThreadData, ErrorInfo]` function that parses a local HTML file using `html.parser.HTMLParser` (stdlib, no BeautifulSoup).
@@ -190,7 +190,7 @@ HOW:
- Return `Result.err(ErrorInfo("ParseError", "fetch_thread_from_html", detail))` on malformed HTML.
VERIFY: `uv run pytest tests/test_twitter_threads_fetch.py` PASSES for the Strategy C tests.
- [ ] **Task 4.3: Implement the URL path (the `gallery-dl` subprocess wrapper)**
- [x] **Task 4.3: Implement the URL path (the `gallery-dl` subprocess wrapper)**
WHERE: `scripts/twitter_threads/fetch_thread.py` (same file; add the URL path)
WHAT: A `fetch_thread_from_url(url: str, cookies_path: Path | None = None) -> Result[ThreadData, ErrorInfo]` function.
@@ -203,7 +203,7 @@ HOW:
- On JSON parse failure: return `Result.err(ErrorInfo("JsonParseError", ...))`.
VERIFY: Manual smoke test against a real X.com URL (the user provides a URL + cookies file). The automated tests cover the HTML path only (Strategy C); the URL path is the network boundary and is smoke-tested manually.
- [ ] **Task 4.4: Implement the CLI dispatch + `--help`**
- [x] **Task 4.4: Implement the CLI dispatch + `--help`**
WHERE: `scripts/twitter_threads/fetch_thread.py` (the `if __name__ == "__main__":` block)
WHAT: A CLI that takes a URL or a local HTML path + `--output` + optional `--cookies`, dispatches to the right function, and writes the `ThreadData` to a JSON file (intermediate) for downstream `download_media.py` + `render_markdown.py` to consume.
@@ -220,7 +220,7 @@ if __name__ == "__main__":
```
VERIFY: `python scripts/twitter_threads/fetch_thread.py --help` works (standalone, no `src/` imports).
- [ ] **Task 4.5: Commit Phase 4**
- [x] **Task 4.5: Commit Phase 4**
```bash
git add scripts/twitter_threads/fetch_thread.py tests/test_twitter_threads_fetch.py