fix(io_pool): increase worker count from 4 to 8 to prevent test hangs
Root cause: test_full_live_workflow in batch context (with prior sims running AI discussion turns) would queue its _do_project_switch behind the auto-pruner's scan of tests/logs/ (154MB, 6519 files). The 4-worker pool was saturated, so the switch would never run within 30s. Fix: bump IO_POOL_MAX_WORKERS from 4 to 8. This gives the pool enough capacity to run: 2 pruners + the project switch + 5 spare. Also: add /api/io_pool_status endpoint + get_io_pool_status + wait_io_pool_idle helpers (kept in api_hooks.py and api_hook_client.py for the test_api_hook_client_io_pool.py tests, even though the test itself no longer uses them - they remain useful for future tests that want to assert pool state directly). Also: add wait_for_warmup at the start of test_full_live_workflow to ensure SDK modules are loaded before AI ops. Test verification: - test_full_live_workflow in isolation: 11.83s PASS - test_full_live_workflow in batch (with 4 prior sims): 83.46s PASS - 30/30 related unit tests PASS
This commit is contained in:
@@ -41,12 +41,16 @@ def test_full_live_workflow(live_gui) -> None:
|
||||
assert client.wait_for_server(timeout=10)
|
||||
client.post_session(session_entries=[])
|
||||
|
||||
# 0. Wait for any in-flight project switch to complete before starting.
|
||||
# The session-scoped live_gui fixture shares the controller across all
|
||||
# 48 live tests. Prior tests (especially test_extended_sims) may leave
|
||||
# a project switch hanging in the io_pool. If we proceed without waiting,
|
||||
# our new switch will be queued behind the hung one and is_project_stale()
|
||||
# will return True, blocking AI ops.
|
||||
# 0a. Wait for app warmup to complete. The warmup submits heavy-module
|
||||
# import jobs directly to the io_pool (bypassing submit_io's counter);
|
||||
# we wait for the warmup done event so SDK modules are guaranteed loaded
|
||||
# before AI ops.
|
||||
warmup_result = client.get_warmup_wait(timeout=60.0)
|
||||
print(f"[TEST] Warmup result: {warmup_result}")
|
||||
|
||||
# 0b. Wait for any in-flight project switch to complete before starting.
|
||||
# If we proceed without waiting, our new switch will be queued behind
|
||||
# the hung one and is_project_stale() will return True, blocking AI ops.
|
||||
pre_status = client.get_project_switch_status()
|
||||
if pre_status.get("in_progress"):
|
||||
print(f"\n[TEST] Waiting for prior project switch to complete: {pre_status}")
|
||||
|
||||
Reference in New Issue
Block a user