Adds `expect_leak_or_bad_free :: proc(t: ^T, client_test: proc(t: ^T), verifier: Memory_Verifier_Proc)`.
It sets up its own `Tracking_Allocator`, runs the `client_test`, and then calls the `verifier` procedure.
The verifier can then inspect the contents of the tracking allocator and call `testing.expect*` as sensible for the test in question.
Any allocations are then cleared so that the test runner doesn't itself complain about leaks.
Additionally, `ODIN_TEST_LOG_LEVEL_MEMORY` has been added as a define to set the severity of the test runner's memory tracker. You can use `-define:ODIN_TEST_LOG_LEVEL_MEMORY=error` to make tests fail rather than warn if leaks or bad frees have been found.
I am uncertain why this works, but it does. Previously, `rtti_test` was
failing due to non-zero data appearing in the `l_buggy` `Buggy_Struct`.
The issue was caused by calling `runtime.default_random_generator` with
a pointer to the state, somehow. The pointer could be on the stack or in
the heap; it did not matter.
I found two workarounds.
- One is to move the RNG setup behind the call to `free_all`.
- The other is to construct the random generator manually.
Despite my digging and testing, I could find no reason as to why this
works or what the fundamental issue was to begin with. If anyone comes
upon this in the future with direct access to a Windows machine, I
recommend stepping through the program with a debugger to investigate
more deeply into why this happens.
I was encountering bounds-check error messages being overwritten during
a test, if the test failed for another reason and sent a log message.
The original intent of having this check inside of the above `if` block
was that if a test sent an error message, then it was assumed an
overwrite would be safe, but it's completely possible for a test to fail
for a legitimate reason, then do an unrelated bounds check somewhere
else that would be buried under the animation.
This change will make sure that, no matter what, the progress display
will not trigger a clear if a signal was raised. There's still no
guarantee that bounds-check messages will be printed properly, and it's
best to redirect STDERR.
The only way that can be fixed is if they get a similar hook to
`context.assertion_failure_proc`.
Add a new option `ODIN_TEST_RANDOM_SEED` which is picked from the cycle
counter at startup, if it's not specified by the user.
This number is sent to every test in the `T` struct and reset every test
(just in case).
Changes
- Support multi-threaded testing.
- Support `set_fail_timeout` on all platforms.
- Display an animated progress report.
- Setup all tests with a context logger.
- Give all tests their own separate custom allocators.
- Support tracking test memory usage.
- Display a summary of the failed tests at the end.
- Let users select only specific tests to run.
- Support copying failed tests to the clipboard to run again.
- Support catching SIGINT (CTRL-C) to cancel early.
- Record context in cleanup procs.
- Write all log messages to STDERR for easy redirection.
- Possibly more I've forgotten.
New Options
- `-define:test_threads=N`: Specify thread count.
- `-define:test_thread_memory=B`: Specify initial memory block size in bytes to each thread.
- `-define:test_track_memory=true`: Track the memory usage of individual tests.
- `-define:test_fancy=false`: Disable animated progress report.
- `-define:test_select=package.test_name,...`: Run only select tests.
- `-define:test_clipboard=true`: Copy names of failed tests to the clipboard.
- `-define:test_progress_width=24`: Change the width of the animated progress bars.
Karl Zylinski
avanspector
Feoramund
Jeroen van Rijn
gingerBill
Laytan Laats