Is the package suitable to do simulations "as fast as possible"?

[stakaz]

Hello, I wounder if this is the use case for this package: modelling diseases and vaccination processes with localization and various different sub-environment, e.g. farms, vaccination teams, wind and whether and so on.

The question is, if I would like to simulate one run of the corresponding "world" as fast as possible, e.g. for training the agents with RL and so on, how would I proceed? Like, how to make everything fast but still use the advantage of the communication protocol and other cool features of this package?

Or is this use case not suited for the use of this package?

Maybe a simple example of the real RL code around e.g. the 2D grid example (like, how to start and stop the environment at each iteration to perform the learning cycles, how to record the results and so on) would be great?

And thanks for fixing the "real_time_factor" so quickly ;)

[wouterwln]

Hey, thanks for trying out RxEnvironments! I think what you want is possible, and is what I call a "discrete time environment", a small section in the documentation can be found here. What this essentially does is that we, instead of using the machine clock to determine time, we fire the update! function whenever your agent emits an action. This is indeed a bit like the Windy Gridworld example as well, only that setup does not have a clock (see the note on that documentation page).

So you could use this package to create a composite environment for RL as well, I think the package is suitable for that. As for the example with an actual learning cycle, this will become available at some point, but I do not have a specific timeframe in mind.

[stakaz]

Thanks for the answer. I will take a look to it at a later point in time. However, possible this is not exactly not what I would need, since I want to have e.g. diseases to come at different (and random) time steps, which would not be possible without an internal clock.

So, basically what I think of is probably a discrete time but with notion of time difference and not just single steps :)