stop locking playwright to specific minor version

[Georgegriff]

Hello!

I notice that you have your playwright version locked to a specific version... i know that this was because playwright made a breaking change to how browsers download in a minor version.

But would you be open to no-longer doing this in a future release?
i recently made a change specifically to help us with artillery testing in playwright and currently we have to force override the playwright version artillery uses

[hassy]

Hi @Georgegriff 👋 The reason we're locking to a specific version is to guarantee that the version of Playwright used in tests on Fargate is the same as the one used locally. How are you force-overriding the version at the moment?

[Georgegriff]

Thanks for the reply, the plan to do it via package.json resolutions, which I thought we'd done previously but maybe not. I i'll be trying it today though.

Some context:
We have architected our tests so we can entirely re-use our core interactions page object models (clicking buttons etc) from our standard playwright tests and artillery ones. It is almost really excellent we have just hit some pain points in differences when using the building playwright/test runner vs playwright core which artillery uses.

[hassy]

We have architected our tests so we can entirely re-use our core interactions page object models (clicking buttons etc) from our standard playwright tests and artillery ones.

Love the sound of this! 😄 We're currently on 1.39.0 which is behind Playwright's latest 1.41. Are the painpoints due to something not being available in the older version? We'll be able to ship a canary release using the latest version of Playwright very quickly if so (and include it in the next mainline release in ~2 weeks).

[Georgegriff]

We have architected our tests so we can entirely re-use our core interactions page object models (clicking buttons etc) from our standard playwright tests and artillery ones.

Love the sound of this! 😄 We're currently on 1.39.0 which is behind Playwright's latest 1.41. Are the painpoints due to something not being available in the older version? We'll be able to ship a canary release using the latest version of Playwright very quickly if so (and include it in the next mainline release in ~2 weeks).

Yes, there's a few features/fixes, like the one i linked above we the expect timeouts from 1.41

[hassy]

PR to update Playwright here: #2425

Once all tests pass we'll be able to merge to main which will produce a new canary release you'll be able to try with npm install -g artillery@canary

[Georgegriff]

Thank you!

[hassy]

hi @Georgegriff - you can try the canary release now with Playwright 1.41.0:

npm install -g artillery@2.0.4-a0b6328

And to run a test on Fargate, you'll need to tell Artillery to use the updated Docker image:

WORKER_IMAGE_URL=public.ecr.aws/d8a4z9o5/artillery-worker:a0b6328719a1223118c362ab923476b97f5a1e83 artillery run-fargate myscript.yml

All built from the most recent commit on main here: a0b6328

[Georgegriff]

This looks to be working, I think technically i don't actually need the upgrade but it's good to have.

The reason it's not 100% needed this time:

The bit of code that we changed in playwright to help us with artillery is expect this, actually comes from @playwright/test, although the code underlying is actually in playwright core.

The reason we're using this is all of our existing page object models use this.

I suspect the reason this is all working happily without having to use your image is:

• playwright core (the artillery version) is being used to run the tests
• we reach into @playwright/test in our JS code just for expect (which we have configured to support the timeout settings outside of the test runner expect.configure({...}))

Because of this, I think we can technically run a different version just for the expects but it is a bit sketchy, so thanks for making this change.

[hassy]

Okay sounds good! We needed to update the Playwright version in Artillery anyway :)