Skip to content

Commit

Permalink
Upgrade Playwright to 1.30.0 (#48007)
Browse files Browse the repository at this point in the history
* Upgrade Playwright to 1.30.0

* Update doc for best practices

* Fix snapshot path

* Fix strict locator

* Try to fix safari test

* Update image test to use visual testing

* Update docs/contributors/code/e2e/README.md

Co-authored-by: Glen Davies <glendaviesnz@users.noreply.github.com>

* Drop unnecessary await in toMatchSnapshot

Co-authored-by: Bart Kalisz <bartlomiej.kalisz@gmail.com>

---------

Co-authored-by: Glen Davies <glendaviesnz@users.noreply.github.com>
Co-authored-by: Bart Kalisz <bartlomiej.kalisz@gmail.com>
  • Loading branch information
3 people authored and ntsekouras committed Feb 21, 2023
1 parent 42bba6e commit be68ed7
Show file tree
Hide file tree
Showing 13 changed files with 110 additions and 108 deletions.
20 changes: 9 additions & 11 deletions docs/contributors/code/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ xvfb-run -- npm run test:e2e:playwright -- --project=webkit

## Best practices

Read the [best practices](https://playwright.dev/docs/best-practices) guide for Playwright.

<details>
<summary><h3>Forbid `$`, use `locator` instead</h3></summary>

Expand All @@ -45,26 +47,22 @@ In fact, any API that returns `ElementHandle` is [discouraged](https://playwrigh
<details>
<summary><h3>Use accessible selectors</h3></summary>

Use the selector engine [role-selector](https://playwright.dev/docs/selectors#role-selector) to construct the query wherever possible. It enables us to write accessible queries without having to rely on internal implementations. The syntax should be straightforward and looks like this:
Use [`getByRole`](https://playwright.dev/docs/locators#locate-by-role) to construct the query wherever possible. It enables us to write accessible queries without having to rely on internal implementations.

```js
// Select a button with the accessible name "Hello World" (case-insensitive).
page.locator( 'role=button[name="Hello World"i]' );

// Using short-form API, the `name` is case-insensitive by default.
// Select a button which includes the accessible name "Hello World" (case-insensitive).
page.getByRole( 'button', { name: 'Hello World' } );
```

It's recommended to append `i` to the name attribute to match it case-insensitively wherever it makes sense. It can also be chained with built-in selector engines to perform complex queries:
It can also be chained to perform complex queries:

```js
// Select a button with a name ends with `Back` and is visible on the screen.
page.locator( 'role=button[name=/Back$/] >> visible=true' );
// Select a button with the (exact) name "View options" under `#some-section`.
page.locator( 'css=#some-section >> role=button[name="View options"]' );
// Select an option with a name "Buttons" under the "Block Library" region.
page.getByRole( 'region', { name: 'Block Library' } )
.getByRole( 'option', { name: 'Buttons' } )
```

See the [official documentation](https://playwright.dev/docs/selectors#role-selector) for more info on how to use them.
See the [official documentation](https://playwright.dev/docs/locators) for more info on how to use them.
</details>

<details>
Expand Down
66 changes: 33 additions & 33 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@
"@octokit/rest": "16.26.0",
"@octokit/types": "6.34.0",
"@octokit/webhooks-types": "5.6.0",
"@playwright/test": "1.27.1",
"@playwright/test": "1.30.0",
"@pmmmwh/react-refresh-webpack-plugin": "0.5.2",
"@storybook/addon-a11y": "6.5.7",
"@storybook/addon-actions": "6.5.7",
Expand Down
18 changes: 0 additions & 18 deletions packages/e2e-test-utils-playwright/src/test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -151,24 +151,6 @@ const test = base.extend<
},
{ scope: 'worker', auto: true },
],
// An automatic fixture to configure snapshot settings globally.
snapshotConfig: [
async ( {}, use, testInfo ) => {
// A work-around to remove the default snapshot suffix.
// See https://github.com/microsoft/playwright/issues/11134
testInfo.snapshotSuffix = '';
// Normalize snapshots into the same `__snapshots__` folder to minimize
// the file name length on Windows.
// See https://github.com/WordPress/gutenberg/issues/40291
testInfo.snapshotDir = path.join(
path.dirname( testInfo.file ),
'__snapshots__'
);

await use();
},
{ auto: true },
],
} );

export { test, expect };
9 changes: 5 additions & 4 deletions test/e2e/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,13 @@
import os from 'os';
import path from 'path';
import { fileURLToPath } from 'url';
import { devices } from '@playwright/test';
import type { PlaywrightTestConfig } from '@playwright/test';
import { defineConfig, devices } from '@playwright/test';

const STORAGE_STATE_PATH =
process.env.STORAGE_STATE_PATH ||
path.join( process.cwd(), 'artifacts/storage-states/admin.json' );

const config: PlaywrightTestConfig = {
const config = defineConfig( {
reporter: process.env.CI
? [ [ 'github' ], [ './config/flaky-tests-reporter.ts' ] ]
: 'list',
Expand All @@ -23,6 +22,8 @@ const config: PlaywrightTestConfig = {
reportSlowTests: null,
testDir: fileURLToPath( new URL( './specs', 'file:' + __filename ).href ),
outputDir: path.join( process.cwd(), 'artifacts/test-results' ),
snapshotPathTemplate:
'{testDir}/{testFileDir}/__snapshots__/{arg}-{projectName}{ext}',
globalSetup: fileURLToPath(
new URL( './config/global-setup.ts', 'file:' + __filename ).href
),
Expand Down Expand Up @@ -81,6 +82,6 @@ const config: PlaywrightTestConfig = {
grepInvert: /-firefox/,
},
],
};
} );

export default config;
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

1 comment on commit be68ed7

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flaky tests detected in be68ed7.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/4231805626
📝 Reported issues:

Please sign in to comment.