ed
e1c8730f20
fix(tests): bound run_tests_batched.py hang at 30s via daemon watchdog
...
run_tests_batched.py hangs at the end of a batch when the pytest
subprocess fails to exit cleanly. Two hang chains have been observed:
1. ThreadPoolExecutor.__del__ -> shutdown(wait=True) joining a
blocked worker during interpreter finalization
(concurrent.futures._python_exit, pool __del__, etc.).
2. The session-scoped \live_gui\ fixture teardown hanging in
client.reset_session() (HTTP call to hook server) or
kill_process_tree(process.pid) / process.wait(timeout=2)
(waiting for the sloppy.py subprocess to die on Windows).
A previous atexit-based fix (commit 8957c9a5 ) attempted to preempt
chain #1 , but verified empirically that atexit handlers do NOT fire
at all when a pool worker is blocked in user code (see
src/io_pool.py module docstring for the full analysis). The
atexit-based fix is therefore ineffective, and was removed from
the conftest in this commit.
Solution: a daemon-thread watchdog that unconditionally calls
os._exit(0) after 30s. If pytest exits cleanly first, the thread
is killed when the process tears down (daemon=True). If pytest
hangs, the watchdog kicks in and the batched runner can move to
the next batch. Same pattern as
src/app_controller.py:_install_sigint_exit_handler (the production
Ctrl+C fix); the difference is the trigger (time-based vs. SIGINT).
Files:
- tests/conftest.py: replaced the ineffective atexit-based fix
with the daemon-thread watchdog. Header comment documents both
hang chains and explains why atexit was abandoned.
- tests/test_conftest_watchdog.py: 3 static regression tests that
verify the watchdog is registered as a daemon thread with a
timeout in the 25-35s range. Static checks (not subprocess) so
the test itself isn't recursively bound by the watchdog.
2026-06-07 10:02:07 -04:00
..
2026-03-02 22:56:32 -05:00
2026-05-10 15:38:16 -04:00
2026-06-04 22:31:22 -04:00
2026-03-02 22:56:32 -05:00
2026-06-07 10:02:07 -04:00
2026-05-07 15:02:00 -04:00
2026-05-04 21:52:39 -04:00
2026-05-14 22:23:48 -04:00
2026-05-13 22:35:52 -04:00
2026-03-05 17:13:59 -05:00
2026-03-05 14:04:17 -05:00
2026-05-06 13:48:47 -04:00
2026-05-09 17:44:15 -04:00
2026-03-07 12:13:08 -05:00
2026-05-16 03:00:20 -04:00
2026-03-06 12:59:18 -05:00
2026-05-13 22:35:52 -04:00
2026-06-06 15:11:13 -04:00
2026-05-09 15:00:35 -04:00
2026-03-11 23:14:09 -04:00
2026-03-05 19:21:57 -05:00
2026-05-12 19:06:54 -04:00
2026-03-12 19:28:21 -04:00
2026-06-06 22:02:35 -04:00
2026-03-11 23:04:42 -04:00
2026-03-12 15:33:37 -04:00
2026-06-06 16:34:46 -04:00
2026-05-13 22:35:52 -04:00
2026-06-07 02:00:56 -04:00
2026-06-02 21:34:56 -04:00
2026-05-16 03:00:20 -04:00
2026-05-16 03:00:20 -04:00
2026-06-02 17:30:46 -04:00
2026-05-10 15:14:54 -04:00
2026-05-20 07:51:58 -04:00
2026-05-13 22:35:52 -04:00
2026-06-06 14:22:18 -04:00
2026-05-16 11:22:41 -04:00
2026-05-08 22:01:15 -04:00
2026-03-06 12:48:02 -05:00
2026-05-06 13:48:47 -04:00
2026-03-10 10:25:09 -04:00
2026-03-10 09:53:59 -04:00
2026-03-10 09:27:12 -04:00
2026-06-03 00:52:08 -04:00
2026-05-13 22:35:52 -04:00
2026-03-04 10:01:55 -05:00
2026-06-02 22:41:59 -04:00
2026-06-02 22:54:52 -04:00
2026-06-06 16:48:04 -04:00
2026-06-03 14:25:59 -04:00
2026-06-03 14:53:05 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-06-06 13:30:18 -04:00
2026-05-06 20:37:48 -04:00
2026-06-07 10:02:07 -04:00
2026-05-16 03:00:20 -04:00
2026-05-16 11:22:41 -04:00
2026-05-16 03:00:20 -04:00
2026-05-16 03:00:20 -04:00
2026-05-11 20:03:01 -04:00
2026-05-20 07:51:58 -04:00
2026-05-20 07:51:58 -04:00
2026-03-22 12:59:10 -04:00
2026-05-16 11:22:41 -04:00
2026-05-16 17:58:19 -04:00
2026-03-06 17:05:48 -05:00
2026-03-05 16:37:30 -05:00
2026-05-11 17:15:04 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-16 03:00:20 -04:00
2026-06-06 18:30:44 -04:00
2026-06-02 01:42:41 -04:00
2026-06-03 11:27:04 -04:00
2026-05-09 15:00:35 -04:00
2026-06-03 11:27:04 -04:00
2026-06-03 08:33:10 -04:00
2026-05-09 12:35:58 -04:00
2026-03-06 12:48:02 -05:00
2026-05-13 22:35:52 -04:00
2026-05-07 21:21:05 -04:00
2026-05-16 03:00:20 -04:00
2026-03-12 15:58:36 -04:00
2026-05-16 03:00:20 -04:00
2026-03-12 15:41:01 -04:00
2026-05-13 22:35:52 -04:00
2026-06-03 12:24:51 -04:00
2026-05-20 07:51:58 -04:00
2026-05-16 03:00:20 -04:00
2026-03-06 12:48:02 -05:00
2026-05-16 03:00:20 -04:00
2026-03-06 12:48:02 -05:00
2026-03-06 12:48:02 -05:00
2026-05-10 11:43:50 -04:00
2026-05-13 22:35:52 -04:00
2026-06-03 12:41:13 -04:00
2026-05-10 15:38:16 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-06-06 17:16:53 -04:00
2026-05-16 14:32:38 -04:00
2026-03-13 12:05:49 -04:00
2026-05-06 13:48:47 -04:00
2026-05-13 22:35:52 -04:00
2026-06-03 11:27:04 -04:00
2026-03-12 19:28:21 -04:00
2026-06-03 11:27:04 -04:00
2026-05-16 15:59:40 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-06-05 10:33:38 -04:00
2026-06-05 10:25:25 -04:00
2026-05-13 22:35:52 -04:00
2026-05-16 03:00:20 -04:00
2026-06-03 11:27:04 -04:00
2026-05-16 14:45:06 -04:00
2026-06-02 17:30:46 -04:00
2026-06-02 02:20:07 -04:00
2026-05-13 22:35:52 -04:00
2026-06-02 17:30:46 -04:00
2026-06-06 21:29:03 -04:00
2026-05-16 01:21:27 -04:00
2026-03-12 19:28:21 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-06-02 01:27:32 -04:00
2026-05-16 03:00:20 -04:00
2026-05-05 17:50:55 -04:00
2026-05-09 15:00:35 -04:00
2026-06-01 17:56:24 -04:00
2026-05-16 03:00:20 -04:00
2026-05-12 19:02:30 -04:00
2026-06-07 02:00:56 -04:00
2026-05-13 22:35:52 -04:00
2026-06-07 01:34:48 -04:00
2026-06-07 02:02:41 -04:00
2026-05-16 01:21:27 -04:00
2026-06-03 17:37:44 -04:00
2026-05-13 22:35:52 -04:00
2026-06-03 13:47:08 -04:00
2026-06-03 11:28:36 -04:00
2026-05-16 14:45:06 -04:00
2026-03-06 12:48:02 -05:00
2026-03-07 12:43:29 -05:00
2026-05-09 15:00:35 -04:00
2026-03-06 16:43:11 -05:00
2026-06-06 18:01:39 -04:00
2026-03-07 16:25:44 -05:00
2026-06-03 21:48:12 -04:00
2026-06-06 16:58:32 -04:00
2026-06-04 23:13:33 -04:00
2026-06-03 21:14:16 -04:00
2026-06-03 11:08:58 -04:00
2026-06-03 21:33:47 -04:00
2026-06-03 10:59:57 -04:00
2026-05-06 13:48:47 -04:00
2026-03-12 15:31:10 -04:00
2026-03-05 17:13:59 -05:00
2026-05-05 19:48:38 -04:00
2026-05-10 14:58:29 -04:00
2026-05-09 15:00:35 -04:00
2026-05-09 15:00:35 -04:00
2026-05-16 14:45:06 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-16 03:00:20 -04:00
2026-05-16 14:45:06 -04:00
2026-06-06 13:30:18 -04:00
2026-05-16 03:00:20 -04:00
2026-05-16 03:00:20 -04:00
2026-03-05 16:37:30 -05:00
2026-03-07 20:32:59 -05:00
2026-05-13 22:35:52 -04:00
2026-03-06 12:48:02 -05:00
2026-03-02 13:26:20 -05:00
2026-06-07 10:01:40 -04:00
2026-06-06 21:42:08 -04:00
2026-03-06 22:03:59 -05:00
2026-03-06 16:43:11 -05:00
2026-03-05 17:13:59 -05:00
2026-05-13 17:22:34 -04:00
2026-03-06 16:55:45 -05:00
2026-05-13 22:35:52 -04:00
2026-03-07 00:15:06 -05:00
2026-03-12 19:08:51 -04:00
2026-03-07 16:41:47 -05:00
2026-05-06 14:59:34 -04:00
2026-05-06 14:59:34 -04:00
2026-05-06 14:30:22 -04:00
2026-03-10 11:09:11 -04:00
2026-03-10 23:21:14 -04:00
2026-03-10 23:21:14 -04:00
2026-03-12 19:28:21 -04:00
2026-05-10 15:46:53 -04:00
2026-05-04 18:49:18 -04:00
2026-03-07 16:36:04 -05:00
2026-05-09 15:00:35 -04:00
2026-05-09 15:00:35 -04:00
2026-05-09 15:00:35 -04:00
2026-06-03 15:18:18 -04:00
2026-06-06 01:12:29 -04:00
2026-05-13 22:35:52 -04:00
2026-03-07 11:24:05 -05:00
2026-05-06 13:48:47 -04:00
2026-03-05 15:10:53 -05:00
2026-03-12 19:08:51 -04:00
2026-05-09 15:00:35 -04:00
2026-05-16 15:59:40 -04:00
2026-06-06 20:19:50 -04:00
2026-05-09 15:00:35 -04:00
2026-05-16 15:59:40 -04:00
2026-06-06 13:21:43 -04:00
2026-05-16 15:59:40 -04:00
2026-05-16 01:21:27 -04:00
2026-06-05 12:38:47 -04:00
2026-05-20 07:51:58 -04:00
2026-05-14 23:38:23 -04:00
2026-05-13 22:35:52 -04:00
2026-05-09 17:36:07 -04:00
2026-05-16 15:59:40 -04:00
2026-05-16 14:45:06 -04:00
2026-03-08 20:51:27 -04:00
2026-03-12 19:08:51 -04:00
2026-03-06 16:43:11 -05:00
2026-05-16 14:45:06 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-05-13 22:35:52 -04:00
2026-03-07 11:54:11 -05:00
2026-05-11 18:19:50 -04:00
2026-03-06 12:48:02 -05:00
2026-06-06 13:57:26 -04:00
2026-05-15 17:17:05 -04:00
2026-05-04 04:52:40 -04:00
2026-05-04 05:20:03 -04:00
2026-03-07 15:20:39 -05:00
2026-05-15 17:17:05 -04:00
2026-05-09 15:00:35 -04:00
2026-05-09 15:00:35 -04:00
2026-03-19 19:53:09 -04:00
2026-05-02 18:31:36 -04:00
2026-05-13 22:35:52 -04:00
2026-05-16 14:45:06 -04:00
2026-03-09 23:16:02 -04:00
2026-06-06 16:55:20 -04:00
2026-06-04 22:31:22 -04:00
2026-03-09 00:55:09 -04:00
2026-03-11 23:49:23 -04:00
2026-03-09 01:13:23 -04:00
2026-06-04 23:16:21 -04:00
2026-05-16 15:59:40 -04:00
2026-05-16 03:00:20 -04:00
2026-03-13 21:55:29 -04:00
2026-05-06 20:37:48 -04:00
2026-05-16 03:00:20 -04:00
2026-06-06 16:09:16 -04:00
2026-05-16 03:00:20 -04:00
2026-05-10 10:13:56 -04:00
2026-03-06 12:59:18 -05:00
2026-05-16 15:59:40 -04:00
2026-05-02 13:23:00 -04:00
2026-03-10 09:53:59 -04:00
2026-03-11 20:30:09 -04:00
2026-05-09 15:00:35 -04:00
2026-03-10 09:27:12 -04:00
2026-03-10 09:27:12 -04:00
2026-03-10 01:23:57 -04:00
2026-05-13 22:35:52 -04:00
2026-05-09 15:00:35 -04:00
2026-05-13 22:35:52 -04:00
2026-05-20 07:51:58 -04:00
2026-05-20 07:51:58 -04:00
2026-05-04 05:18:04 -04:00
2026-05-16 11:22:41 -04:00
2026-05-09 16:55:45 -04:00
2026-03-09 23:25:06 -04:00
2026-03-07 21:40:40 -05:00
2026-06-03 11:31:29 -04:00
2026-06-05 11:52:49 -04:00
2026-05-09 16:55:45 -04:00
2026-05-16 03:00:20 -04:00
2026-06-02 02:20:07 -04:00
2026-05-13 22:35:52 -04:00
2026-03-04 09:46:02 -05:00
2026-06-06 22:48:50 -04:00
2026-06-06 14:47:02 -04:00
2026-05-20 07:51:58 -04:00
2026-03-04 10:01:55 -05:00
2026-05-05 20:51:03 -04:00
2026-06-05 20:14:39 -04:00
2026-05-13 22:35:52 -04:00
2026-05-14 23:13:17 -04:00