Private
Public Access
0
0
Commit Graph

5037 Commits

Author SHA1 Message Date
ed be93c262e0 test: delete 3 trivial/deprecated test files (t2.3 partial)
- test_phase_3_final_verify.py: 6-line tautology that imports
  verify_phase_3 from a now-deleted verification module.
- test_mma_skeleton.py: imports generate_skeleton from the deprecated
  scripts.mma_exec module (replaced by OpenCode Task tool per workflow.md).
- test_arch_boundary_phase1.py: tests hardcoded-path detection on
  deprecated scripts/mma_exec.py + scripts/claude_mma_exec.py.

These were either no-ops or testing now-deprecated infrastructure.
Kept test_arch_boundary_phase2.py + test_arch_boundary_phase3.py
(more recent phase verifications not tied to deprecated modules).
2026-07-05 20:14:49 -04:00
ed 6a3c142bfa test: migrate src.models bare imports + PROVIDERS/Metadata sub-imports to direct subsystems
Removed 18 unused 'from src import models' imports. Migrated the
explicit sub-imports:
- 'from src.models import PROVIDERS' -> 'from src.ai_client import PROVIDERS'
  (test_minimax_provider x2, test_deepseek_infra)
- 'from src.models import Metadata' -> 'from src.type_aliases import Metadata'
  (test_project_manager_tracks, test_track_state_persistence, test_track_state_schema)

Kept the 2 'import src.models as models' in test_provider_curation +
test_providers_source_of_truth (they verify the backward-compat shim
still re-exports PROVIDERS) and the 2 self-tests
(test_models_no_top_level_pydantic + test_models_no_top_level_tomli_w)
that exercise the shim's no-leak invariants.
2026-07-05 20:12:17 -04:00
ed bd6fc3e259 test: delete 9 dead tests (2 chronology + 7 scavenge) + add test_directive_structure.py
The 2 tests/test_*chronology* files imported from scripts.audit.* modules
that were deleted 2026-07-05 (Tier 1 manual-chronology maintenance); both
were ImportError on collection.

The 7 test_scavenge_*.py files were one-shot verifications of the
2026-07-02/03 directive-lift (specific names exist) — permanently green
tautologies with zero ongoing value.

test_directive_structure.py enforces the general invariant every
directive in conductor/directives has v1.md + meta.md + matching
heading.
2026-07-05 20:10:22 -04:00
ed 8dfc53447d conductor(track): state.toml at tier2 mid-checkpoint (t1.9 partial, 8 commits)
Front A source edits (t1.1-t1.6): all clean. PROVIDERS=7; adapter +
7 dedicated test files deleted. t1.9 minor edits: 14 sites across 13
test files cleaned (provider list expectations, _update_gcli_adapter
monkeypatches, ui_gemini_cli_path setters, comment references).
Remaining: t1.7 (mock provider implementation), t1.8 (11 sim test
rewrites, blocked on t1.7), t1.10 (12 docs), t1.11 (batch checkpoint).
Phases 2 and 3 untouched.
2026-07-05 20:02:41 -04:00
ed 2811cc230b test: remove stale ui_gemini_cli_path from test_view_presets
ui_gemini_cli_path was removed from AppController.__init__ in t1.3;
this stray setter survived in the test fixture scaffold (no behavioral
test value).
2026-07-05 20:02:17 -04:00
ed 9401d3f67d test: more t1.9 minor edits (gcli_adapter patches, comments)
Drop the remaining _update_gcli_adapter monkeypatches from
test_ai_loop_regressions_20260614 (3 sites), test_live_gui_integration_v2
(2 sites), and the gemini_cli_adapter comment references in
tests/test_ai_client_tool_loop_send_func.py, test_lazymodule_filedialog_fallback.py,
and mock_concurrent_mma.py.
2026-07-05 20:00:20 -04:00
ed edebeac619 test: remove gemini_cli references in 6 + delete test_ai_client_list_models.py
Drop stale gemini_cli expectations: PROVIDERS expectations in
test_providers_source_of_truth + test_provider_curation, the
test_list_models_gemini_cli function (deleted the whole file since it
was the only test), the test_discussion_compression_gemini_cli function,
the test_gcli_path_updates_adapter function (its setter was removed),
the ui_gemini_cli_path references in test_mma_tier_usage_reset_fix and
test_rag_integration. These are the t1.9 minor edits.
2026-07-05 19:56:57 -04:00
ed 832a030500 conductor(track): update state.toml at tier2 checkpoint (t1.1-t1.6 done)
phase_1.status = in_progress_partial_6_of_11; tasks 1.1-1.6 marked completed with SHAs. Tasks 1.7 (mock provider) through 1.11 (batch run) remain pending. Phases 2 and 3 entirely untouched. current_phase advanced from 0 to 1 to reflect Front A mid-execution.
2026-07-05 19:53:28 -04:00
ed 7e06d812d4 conductor(plan): mark Phase 1 tasks 1.1-1.6 as complete (Front A core deletions)
Tasks 1.1-1.6 cover the bulk of Front A: the gemini_cli adapter module
plus all importer edits (ai_client, app_controller, gui_2,
project_manager, api_hooks) and the 7 dedicated gemini_cli test files.
Remaining Front A tasks (1.7-1.11) cover the mock provider, the 11 sim
test rewrites, the minor test edits, the doc updates, and the batch
checkpoint.
2026-07-05 19:51:17 -04:00
ed a93a61fd77 refactor(app_controller): remove gemini_cli state and config
Drop the 11 gemini_cli sites: the ui_gemini_cli_path state field, the
gcli_path settable map entry, the _update_gcli_adapter method, the
project load/save gemini_cli binary_path entries, and the per-provider
gemini_cli guard at startup.
2026-07-05 19:50:15 -04:00
ed 2bba7f56e3 refactor(ai_client): remove gemini_cli provider from ai_client
Drop the standalone Gemini CLI adapter from the AI client surface: delete
the import, the PROVIDERS entry, the module state, the 3 functions
(_list_gemini_cli_models, _send_cli_round_result, _send_gemini_cli), and
the 8 dispatch branches. PROVIDERS now has 7 entries; _gemini_sdk
remainder is unaffected.
2026-07-05 19:49:37 -04:00
ed 9a308c36dc Merge remote-tracking branch 'tier2-clone/tier2/twitter_threads_extraction_20260705' 2026-07-05 19:34:31 -04:00
ed ba1dbd326f conductor(track): init test_suite_cleanup_gemini_cli_removal_20260705 — 3-front cleanup track
3 fronts, 3 phases, 31 tasks, 12 verification criteria. Based on 3
parallel explore-agent audits of the live codebase.

Front A — Gemini CLI Adapter Removal: delete src/gemini_cli_adapter.py
+ 7 test files; edit ai_client.py (3 functions + 8 dispatch branches +
PROVIDERS 8->7); edit app_controller.py (~11 sites) + gui_2.py (2 blocks)
+ project_manager.py + api_hooks.py; introduce a mock provider that
routes to tests/mock_gemini_cli.py so the 9 live_gui sim tests switch
from gemini_cli to mock without losing their mock-LLM mechanism; update
7 docs + 5 conductor docs.

Front B — Test Suite Cleanup (410 files, 2,133 tests, ~45-60 files
downsizable): delete 2 guaranteed-broken chronology tests (import from
deleted scripts.audit.*); consolidate 7 test_scavenge_*.py tautologies
into 1 test_directive_structure.py; review deprecated-module tests;
consolidate ~15 test_*_phase*.py one-offs; migrate 27 src.models shim
importers to direct subsystem imports; extract shared mock_controller
fixture from ~8 boilerplate-patch files; fix 3 fix-not-skip candidates
(Gemini 503 in summarize.summarise_file); audit the 120KB
test_gui_2_result.py (9.4% of suite line count in one file).

Front C — Metadata-Type Reduction: flip the 38-site MCP dispatch
inversion (dispatch()+22 consumers still call str wrappers, not the
_result variants); migrate app_controller.py's ~40 in-memory Metadata
state sites to typed per-aggregate dataclasses; fix aggregate.py
file_items: list[Metadata] -> list[FileItem]; delete dead code
(openai_schemas.to_legacy_dict, app_controller._push_mma_state_update).

Out of scope: GEMINI_CLI_HOOK_CONTEXT env var + cli_tool_bridge.py +
mma_exec.py (meta-tooling, not the provider — separate follow-up if
user wants to purge naming).
2026-07-05 19:33:20 -04:00
ed d4e809ba63 conductor(track): close twitter_threads_extraction_20260705 — corpus extracted (VC8 complete) 2026-07-05 19:29:14 -04:00
ed dc99722bf8 saples updated 2026-07-05 19:09:22 -04:00
ed 2c680e23e4 feat(twitter_threads): gallery-dl media download + inline media embeds in Markdown
download_media now shells out to gallery-dl (handles X.com auth/403/video->mp4) instead of urllib, returns the downloaded files, + media_files_for_post glob helper + --cookies CLI. render_markdown embeds images with ![](...) and videos with an HTML <video controls> tag (were plain links). extract_corpus uses the package download_media (one download path). Tests rewritten for the gallery-dl API. Corpus: 4 threads, 33 media (32 img embeds + 1 video), 0 missing.
2026-07-05 19:08:43 -04:00
ed 3f5a2b0659 samples 2026-07-05 18:52:32 -04:00
ed 7e828cbd61 refactor(twitter_threads): consolidate corpus extraction into one script; gallery-dl downloads media
Replaced convert_cookies.py + run_corpus.py + dedupe_corpus.py (+ the scratch merge_corpus.py) with a single extract_corpus.py that: converts cookies, fetches each URL's full conversation, MERGES overlapping conversations into one thread (union of posts, no duplicate threads/media), downloads ALL media (images + video/mp4) via gallery-dl itself (plain urllib 403'd on pbs.twimg.com and never fetched videos), and renders. Result: 4 merged threads, 33 media files (incl mp4), 0 missing links. cookies_netscape.txt gitignored.
2026-07-05 18:50:55 -04:00
ed ae9cc1d494 feat(twitter_threads): full-thread extraction (conversations) + front-matter from target + corpus dedupe
fetch_thread_from_url now passes -o conversations=true (full threads, not single tweets), sorts posts chronologically, and sets root_post_id to the URL's target tweet. render_markdown builds front-matter from the target (root_post_id) post, not the earliest conversation tweet (fixes wrong @handle). Added dedupe_corpus.py: keeps deepest thread per conversation, deletes duplicates, keeps+marks unique branches (branch_of) + writes threads_index.json. 33 tests pass.
2026-07-05 18:38:43 -04:00
ed 53721a7796 fix(twitter_threads): align gallery-dl parser to live schema + add track extraction scripts
Live Task 5.3 validation against real gallery-dl output: (1) pass -o text-tweets=true so text-only tweets are returned (media-only default yielded []); (2) fix author name/nick swap (gallery-dl name=handle, nick=display). Added convert_cookies.py (JSON cookie export -> Netscape) + run_corpus.py (8-URL pipeline -> docs/twitter/) to the track dir. Extracted 8/8 threads. 33 tests still pass.
2026-07-05 18:28:40 -04:00
ed 12bee9aba6 ignore cookies.txt 2026-07-05 18:17:03 -04:00
ed 5e95a1bfb7 fix(reports): duration_days -> active days for all 30 top tracks
Script used calendar-day duration (end_date - init_date), which
inflated tracks that sat dormant between work and archive-move.
Example: external_editor_integration_20260308 reported 60 days
(init 2026-03-08, last archive-move 2026-05-07) when actual work
was 2 active days.

Corrected metric: active days = distinct calendar days with at
least one commit touching the track folder. All 30 top tracks
now show 2-7 active days, matching the user's expectation that
no track lasts more than a week.

Also fixed commit_count + files_touched for all 30 tracks (the
script's work-prefix-only count was always 0-1 for these).

Top 3 examples:
- external_editor_integration_20260308: 60d/1c/4f -> 2d/4c/8f
- code_path_audit_20260607: 18d/1c/7f -> 7d/23c/14f
- nagent_review_20260608: 12d/0c/20f -> 3d/53c/20f

Still stale (need recompute, not hand-fix):
- 3.1 per-era aggregates
- 8 use-case predictions
- 218 other tracks not in top-30
2026-07-05 18:02:15 -04:00
ed f1c4516034 fix(reports): manually correct commit_count + files_touched for 5 spot-checked tracks
The script's commit_count and files_touched columns only counted
work-prefix commits (feat|fix|refactor|perf|test) with pathspecs
matching the track folder. Track folders contain only metadata
commits (conductor(*), chore(conductor), docs(track)), so the
script reported 0 commits for any track that did real work.

Corrected rows (now using total commits + total unique files
touching the track folder, any prefix):

- code_path_audit_20260607: 1->23 commits, 7->14 files
- qwen_llama_grok_integration_20260606: 0->25 commits, 4->8 files
- nagent_review_20260608: 0->53 commits, 4->20 files
- data_oriented_error_handling_20260606: 0->21 commits, 4->8 files
- superpowers_review_20260619: 0->63 commits, 8->16 files

Added 9.1 caveat explaining the manual corrections and the
methodology error that caused them.
2026-07-05 17:54:01 -04:00
ed 0d7293d59e conductor(track): complete twitter_threads_extraction_20260705 + TRACK_COMPLETION report [state=completed]
All 5 phases done, 34 tests passing, VC1-VC7 pass. Task 5.3 (live 8-URL smoke) + 5.6 (user manual verification) deferred to user (network/cookies + merge decision).
2026-07-05 17:53:42 -04:00
ed fe207c1e42 docs(twitter_threads): README — standalone usage + strategies + copy-to-another-repo
FR7/VC2: prerequisites (gallery-dl, cookies.txt, py3.11+), usage for all 3 stages (-m and bare-script), local-HTML fallback, output layout, Markdown schema, URL normalization, strategies A-D, copy-to-another-repo instructions.
2026-07-05 17:50:30 -04:00
ed ed86731701 feat(twitter_threads): thread_from_dict + download_media/render_markdown CLIs (end-to-end -m pipeline)
thread_from_dict (JSON wire boundary) round-trips thread_data.json; download_media/render_markdown gain argparse main() + __main__ so the plan Task 5.3 pipeline runs via -m; dual-import added so all modules run standalone or via -m (G5). Pipeline integration test (html->names->render) + CLI tests cover the glue without network. TDD: red -> green (4 pipeline + 29 regression = 33 passed).
2026-07-05 17:48:07 -04:00
ed 41656d6b7c conductor(plan): mark Phase 4 (fetch_thread.py) complete [06ff9299] 2026-07-05 17:36:37 -04:00
ed 06ff9299e2 feat(twitter_threads): fetch_thread.py — html.parser (local HTML) + gallery-dl subprocess (URL) + URL query-strip + CLI
fetch_thread_from_html parses the data-attr article contract via stdlib html.parser; _normalize_url strips ?s=20 quote-share suffix (spec FR2); fetch_thread_from_url wraps gallery-dl --dump-json (best-effort, manually smoke-tested); dual-import (absolute/relative) makes it runnable via -m AND as a bare script (VC6/G5); CLI writes thread_data.json. TDD: red(no module) -> green(7 fetch + 22 regression = 29 passed).
2026-07-05 17:36:03 -04:00
ed 0206b09009 docs(chronology): add row for meta_tooling_duration_analysis_20260705 archive
- Move conductor/tracks/meta_tooling_duration_analysis_20260705/ to
  conductor/archive/meta_tooling_duration_analysis_20260705/ (preserves
  history as a directory rename since the spec/plan/metadata files were
  uncommitted; they're added to the archive in this commit).
- Prepend the track's row to conductor/chronology.md (newest-first).
- Note: spec.md, plan.md, metadata.json, state.toml are being committed
  for the first time in this archive commit. Per project convention,
  spec + plan are committed at archive time per workflow.md chronology
  maintenance rules.
2026-07-05 17:28:30 -04:00
ed ea33a49f15 conductor(plan): mark Phase 3 (download_media.py) complete [f503eb5d] 2026-07-05 17:25:09 -04:00
ed f503eb5de3 feat(twitter_threads): download_media.py — stdlib urllib + idempotent + typed per-kind naming
media_names_for_post derives <post_id>_<kind><index>.<ext> (img/vid/gif, per-kind counters) from media_urls; download_media(thread, output_dir) -> Result[list[Path]] downloads via stdlib urlopen, idempotent skip when target exists with size>0, URLError -> Result.err(HttpError). Authorized HTTP-boundary mock tests. TDD: red(no module) -> green(6 media + 16 regression pass).
2026-07-05 17:24:45 -04:00
ed 36de54a9e0 conductor(plan): mark Phase 2 (render_markdown.py) complete [ef98f6d1] 2026-07-05 17:17:52 -04:00
ed ef98f6d1a1 feat(twitter_threads): Result[T] + render_markdown.py (YAML front-matter + per-post sections + media links)
Added standalone Result[T] (frozen+slots, ok/err classmethods, is_ok) to error_types.py per canonical AND-over-OR error_handling.md (not the video_analysis _Ok|_Err sum type). render_markdown renders YAML front-matter from root post, per-post ## sections with reply markers, quote blockquotes, and ./media/<name> links; media_names mapping keeps render decoupled from download_media. TDD: red (no Result) -> green (7 render + 9 types passed).
2026-07-05 17:17:18 -04:00
ed ce0d143877 feat(reports): META_TOOLING_DURATION_ANALYSIS_2026-07-05.md (10 sections, 379 lines)
Empirical duration analysis of the meta-tooling that built this
codebase from Feb 21 to Jul 5, 2026 (133 days, 248 tracks, 1003
sub-agent tasks, 1608 work-prefix commits, 312 sessions).

Era detection: 5 eras via multi-signal weighted-vote on
config.toml + conductor/workflow.md + conductor/tech-stack.md +
sub-agent log content (7-day cluster window).

Headline numbers:
- Era 0 (early, Feb 21 - Mar 20): 113 tracks
- Era 1 (mid, Mar 21 - May 3): 4 tracks (small era)
- Era 2 (mid, May 4 - Jun 1): 42 tracks
- Era 3 (late, Jun 2 - Jun 24): 69 tracks (peak productivity)
- Era 4 (current, Jun 25 - present): 20 tracks

Tier 3 worker median task duration: 272.5s (~4.5 min).
Tier 4 QA median task duration: 416s (~7 min).
Tier 1 median task duration: 0s (synthetic/orchestration only).

Use-case predictions (per files-touched bucket):
- 1-5 files (227 tracks): 0d median, 1.0 IQR
- 6-15 files (15 tracks): 1d median, 6.0 IQR
- 16-30 files (4 tracks): 2d median, 4.5 IQR
- 31+ files (2 tracks): 0d median (small sample)

Part of conductor/tracks/meta_tooling_duration_analysis_20260705/.
2026-07-05 17:14:16 -04:00
ed 96c5b94616 perf(audit): single-pass git walk in track indexer (>10x speedup)
The original build_track_index spawned 5+ git subprocesses per track
folder (244 tracks * 5 calls = 1200+ subprocesses). That exceeded the
120s NFR1 budget. Replaced with a single 'git log --all --name-only'
pass + a mutable _TrackAccumulator; reduces git subprocess count from
1200+ to 1.

Also adds 3 smoke tests for build_subagent_task_index,
build_session_index, build_track_index (covering previously untested
pure-function indexers). 23 tests pass; coverage up from 39% to 61%.
2026-07-05 17:13:25 -04:00
ed 0440be0288 conductor(plan): mark Phase 1 (scaffold + error types + dataclasses) complete [161d8da] 2026-07-05 17:08:31 -04:00
ed 161d8da882 feat(twitter_threads): scaffold package + error types + typed dataclasses
TIER-2 READ AGENTS.md, conductor/workflow.md, conductor/edit_workflow.md, conductor/tier2/githooks/forbidden-files.txt, conductor/tracks/tier2_leak_prevention_20260620/spec.md (archived -> read docs/reports/2026-06-15/TRACK_COMPLETION_tier2_leak_prevention_20260620.md), conductor/code_styleguides/data_oriented_design.md, conductor/code_styleguides/python.md, conductor/code_styleguides/type_aliases.md, conductor/code_styleguides/error_handling.md, conductor/product-guidelines.md (Core Value via aggregate_directives) before Phase 1 (scaffold + error_types + dataclasses).

scripts/twitter_threads/__init__.py (namespace docstring); error_types.py (ErrorInfo + make_error + PostMetrics/PostData/ThreadData, frozen+slots); tests/test_twitter_threads_types.py (9 contract tests).
2026-07-05 17:07:04 -04:00
ed 127be22f89 feat(audit): meta-tooling duration analyzer (8 modules, 10-section renderer)
- scripts/audit/analyze_meta_tooling_durations.py (~700 lines)
- tests/test_analyze_meta_tooling_durations.py (20 unit tests, all pass)
- Era detector with multi-signal weighted-vote clustering
- Track indexer reusing git-history-evidence primitives
- 3 op-layer indexers (sub-agent tasks, work-prefix commits, session logs)
- Statistician with per-era aggregations
- 10 Markdown section renderers
- CLI: --json-only, --output-dir, --era-window-days, --date

Strictly Meta-Tooling domain (docs/guide_meta_boundary.md). Zero
src/ changes; zero GUI changes; zero Application-domain impact.

Part of conductor/tracks/meta_tooling_duration_analysis_20260705/.
2026-07-05 17:02:40 -04:00
ed 0908f8fa28 conductor(track): init twitter_threads_extraction_20260705 — standalone Twitter/X thread extraction tooling
Scripts + workflow for extracting Twitter/X posts and threads into
Markdown with associated media. Mirrors the scripts/video_analysis/
pattern. Standalone requirement: zero imports from src/, conductor/,
or scripts.video_analysis — copy-pasteable to another repo with only
gallery-dl as the external dep.

5 modules: __init__.py, error_types.py (Result[T, ErrorInfo] +
ThreadData/PostData typed dataclasses), fetch_thread.py (gallery-dl
subprocess for URLs + html.parser fallback for local HTML),
download_media.py (stdlib urllib, idempotent), render_markdown.py
(YAML front-matter + per-post sections + ./media/ links).

Reference project: C:\projects\forth\bootslop — the corpus feeds
bootslop's scripts and reference-generation pipeline. Acceptance
corpus: 8 threads (@NOTimothyLottes x6 + @VPCOMPRESSB x2) extracted
to tests/artifacts/twitter_threads_corpus/. The ?s=20 quote-share
suffix on the @VPCOMPRESSB URLs must be stripped by fetch_thread.py
before acquisition (added to FR2 as URL normalization).

5 phases / 23 tasks. 8 verification criteria (VC1-VC8). TDD red-first
on the pure-function modules (render_markdown, types, media naming).
2026-07-05 16:50:05 -04:00
ed 4c9fc99cd4 docs(chronology): switch to manual maintenance; delete generator scripts; archive review report
Deleted scripts/audit/generate_chronology.py + chronology_quality_gate.py
after repeated corruption incidents (auto-classifier drifted from
user intent, silently rewrote rows). chronology.md is now maintained
by hand: add a row at the top when a track ships/abandons/is archived.

Updated conductor/workflow.md §Chronology Maintenance and
conductor/tracks.md §Archiving a track to reflect manual maintenance.

Added 40 rows to conductor/chronology.md for the 2026-07-05 archive
batch, with archive-folder paths and deferral notes where applicable.

Removed 12 archived-track rows from conductor/tracks.md Active Tracks
table (rows 2, 3, 7c, 16, 17, 23b, 23c, 23d, 25, 26, 29, 29c) and
their track-detail anchors.

Wrote docs/reports/ARCHIVE_REVIEW_20260705.md: a categorized index
of unfinished work items from the archived tracks' state.toml files
that remain relevant to the codebase, with a ranked follow-up list.
2026-07-05 16:17:13 -04:00
ed 2f3a8283f9 archive: test patrch fixes and sandbox hardening 2026-07-05 15:56:33 -04:00
ed 78204ac392 archive: send result to send 2026-07-05 15:56:19 -04:00
ed 39fcf12a6b archive: post-module taxonomy de cruft 2026-07-05 15:56:10 -04:00
ed f7136c5fff archive: phase2_4_5 call site completion 2026-07-05 15:55:59 -04:00
ed 3493a95144 archive: cruft elimination 2026-07-05 15:55:41 -04:00
ed 5d060a477d archive: fable review 2026-07-05 15:55:30 -04:00
ed aacfb1ff56 archive: context preview fixes 2026-07-05 15:54:39 -04:00
ed f619e05021 archive public api migration 2026-07-05 15:54:28 -04:00
ed fe10892d48 archive: superpowers review tracks 2026-07-05 15:54:18 -04:00
ed 8c98a3bc42 archive: code path audit polish. 2026-07-05 15:53:43 -04:00