Staggered Parallel Process

[iamkenos]

Problem context

• the team im working with at the moment still uses behavex 1.6
• our test harness does an API call to an endpoint with some sort of rate limiting capability; im not quite sure about the exact numbers but the fromt the looks of it, the endpoint is going to block calls from the same IP within a set ms window.
• using behavex 1.6 to run our tests with parallel process enabled is running smooth - no rate limit errors from the API
• now we want to upgrade to the latest version 4.x to use the new and improved features that the behavex has worked on
• with major changes on how parallel process are handled in 4.x, now we're getting rate limit error from the API and is blocking our tests

I'm not quite sure what exactly is the culprit but from a user perspective, one can say that concurrency was working smoother in 1.6 than >= 3.x.

Is your feature request related to a problem? Please describe.
We would like to have the ability to optionally stagger spinning up threads during parallel execution.

Describe the solution you'd like
I'm thinking of an optional --parallel-staggered-start or something similar that accepts a value in milliseconds.
Internally, this value (when present) will be used to stagger the process spinup. if not present, handle as-is.

Describe alternatives you've considered
We already installed a retry feature on the API request via a module called backoff but that didnt really help so much.

Hope you can consider this request and thanks for all your work!

[hrcorval]

Hi @iamkenos, I perfectly understand your concerns, as we are changing in v4.x the approach to perform parallel executions, and despite we can start performing more stable parallel executions, the internal behavior is not the same.

We will analyze if there are place for improvements in the way the ProcessPoolExecutor works. In the meantime, I like the idea of adding an execution argument like --parallel-staggered-start to accept a value that enable stagger spinning up processes. That would be also a feature that we consider would be useful for many users (even for us). I'll reply soon indicating if this can be included in upcoming version.

Thanks for giving us visibility on the issues you are experiencing

[anibalinn]

Hi @iamkenos , please, give it a try to the following PR and let us know if this might work for you:
#143

Regards

[iamkenos]

hi @anibalinn @hrcorval thanks for the quick responses and patches.

i'm not sure how to build the thing from the PR branch but i see that there's already one open scheduled for the next release so i'll wait for that and test from the new package if that works?

thanks again! you guys are champs 💪🏻

[hrcorval]

Hi @iamkenos, We did a couple of improvements in the way the parallel processes are being managed, and also we have provided the --parallel-delay argument, that allows users to control the delay between starting parallel processes.
I would suggest you give it a try with latest version, using that one in the same way you are doing with v 1.6.0. If this doesn't work, it might be related to the way that ProcessPoolExecutor launches/manages parallel processes.
If that doesn't work, you can also try with the --parallel-delay argument.

Latest version: https://pypi.org/project/behavex/4.0.5/

We hope it helps!

[iamkenos]

hey gents, thanks for the timely updates. i'll take it for a spin some time this week and will keep you all updated. huge thanks!!

[iamkenos]

hi team, thanks for the patience and support. the parallel stagger helped. we can close this feature request now. 🥇

[hrcorval]

Thanks for your feedback. We are glad to see it worked.

Regards