Private
Public Access
0
0

fix(io_pool): increase worker count from 4 to 8 to prevent test hangs

Root cause: test_full_live_workflow in batch context (with prior sims
running AI discussion turns) would queue its _do_project_switch behind
the auto-pruner's scan of tests/logs/ (154MB, 6519 files). The 4-worker
pool was saturated, so the switch would never run within 30s.

Fix: bump IO_POOL_MAX_WORKERS from 4 to 8. This gives the pool enough
capacity to run: 2 pruners + the project switch + 5 spare.

Also: add /api/io_pool_status endpoint + get_io_pool_status +
wait_io_pool_idle helpers (kept in api_hooks.py and api_hook_client.py
for the test_api_hook_client_io_pool.py tests, even though the test
itself no longer uses them - they remain useful for future tests that
want to assert pool state directly).

Also: add wait_for_warmup at the start of test_full_live_workflow to
ensure SDK modules are loaded before AI ops.

Test verification:
- test_full_live_workflow in isolation: 11.83s PASS
- test_full_live_workflow in batch (with 4 prior sims): 83.46s PASS
- 30/30 related unit tests PASS
This commit is contained in:
2026-06-08 17:49:34 -04:00
parent 9afc93bce2
commit 4a33848620
7 changed files with 168 additions and 10 deletions
+10 -6
View File
@@ -41,12 +41,16 @@ def test_full_live_workflow(live_gui) -> None:
assert client.wait_for_server(timeout=10)
client.post_session(session_entries=[])
# 0. Wait for any in-flight project switch to complete before starting.
# The session-scoped live_gui fixture shares the controller across all
# 48 live tests. Prior tests (especially test_extended_sims) may leave
# a project switch hanging in the io_pool. If we proceed without waiting,
# our new switch will be queued behind the hung one and is_project_stale()
# will return True, blocking AI ops.
# 0a. Wait for app warmup to complete. The warmup submits heavy-module
# import jobs directly to the io_pool (bypassing submit_io's counter);
# we wait for the warmup done event so SDK modules are guaranteed loaded
# before AI ops.
warmup_result = client.get_warmup_wait(timeout=60.0)
print(f"[TEST] Warmup result: {warmup_result}")
# 0b. Wait for any in-flight project switch to complete before starting.
# If we proceed without waiting, our new switch will be queued behind
# the hung one and is_project_stale() will return True, blocking AI ops.
pre_status = client.get_project_switch_status()
if pre_status.get("in_progress"):
print(f"\n[TEST] Waiting for prior project switch to complete: {pre_status}")