Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idea: deterministic thread and I/O handling for certain tests #3089

Open
Zac-HD opened this issue Sep 17, 2024 · 0 comments
Open

Idea: deterministic thread and I/O handling for certain tests #3089

Zac-HD opened this issue Sep 17, 2024 · 0 comments

Comments

@Zac-HD
Copy link
Member

Zac-HD commented Sep 17, 2024

For testing, e.g. with Hypothesis, it can often be useful to run deterministically even at the cost of performance.

Using a MockClock(rate=0, autojump_threshold=0) and setting _ALLOW_DETERMINISTIC_SCHEDULING=True does the trick for tests that only use in-process Trio interfaces, but if you need to test with real network IO or multiple threads, then the variation in timing can change which tasks can be scheduled at each point, and we're nondeterministic again.

I recently saw a neat proof-of-concept that gets around this: whenever you would do real IO (i.e. not controlled by the MockClock), do the real IO but don't let reality control the timing. Instead, use the PRNG to choose how long that operation will take - if it finishes earlier, hold it until the chosen time; if it's not ready by then hold the rest of the program (including the clock) until the operation finishes.

Brief notes:

  • I think we'd want to patch trio.to_thread.run_sync, trio.from_thread.run, trio.from_thread.run_sync, trio.serve_listeners, trio.open_tcp_stream, trio.lowlevel.wait_readable, trio.lowlevel.wait_writable, and trio.open_signal_receiver to support this where possible, or explicitly error out otherwise.
  • It might also be interesting to automatically use an in-process network when a client and server are in the same process, instead of going over real OS sockets. This is very convenient but also I dislike silently testing something other than what the user expects.
  • I think this is too big a change to automatically activate whenever there's a suitable clock and deterministic scheduling is enabled, so we'll want some way to explicitly turn it on, too.

I'm not fully convinced that we should ship something like this, but it seems interesting enough to write up and investigate futher.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant