Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for different keyboard layouts #24249

Closed
wants to merge 28 commits into from
Closed
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
0c04dbc
feat: greek and portuguese keyboard layouts
ruifigueira Jul 16, 2023
d3dc96c
fix: use virtual codes as keyCode output
ruifigueira Jul 16, 2023
87b1763
fix: ENTER key definition text set to '\r'
ruifigueira Jul 16, 2023
41cb771
feat: add default keyboards per locale
ruifigueira Jul 16, 2023
24a5f9e
lint: fix DEPS
ruifigueira Jul 17, 2023
2de5053
refactor: replace keyboard layouts json with ts
ruifigueira Jul 17, 2023
e897c98
fix: some shift keys were missing
ruifigueira Jul 17, 2023
de0644f
feat: apis to set / change keyboard layout
ruifigueira Jul 17, 2023
d3511c2
chore: fix code review
ruifigueira Jul 19, 2023
73f4a14
chore: keyboard layout generator xml parser
ruifigueira Jul 19, 2023
d811483
chore: throw error if keyboard layout name is invalid
ruifigueira Jul 19, 2023
c08b89c
chore: remove all generated keyboard layouts
ruifigueira Jul 19, 2023
254ee8b
docs: update keyboard layouts documentation
ruifigueira Jul 19, 2023
02e5620
chore: keyboard layout doc feeds the generator
ruifigueira Jul 19, 2023
f17d69c
tests: restrict tests to generated keyboard layouts
ruifigueira Jul 19, 2023
67974d9
chore: remove dead code
ruifigueira Jul 19, 2023
74bdf55
fix: remove unused toImpl fixture
ruifigueira Jul 24, 2023
119cb97
chore(input): add keyboard layouts
ruifigueira Jul 24, 2023
030b70f
refactor(input): use code as layout filename and replace locale with …
ruifigueira Jul 27, 2023
b754e8e
chore(input): add clean flag for keyboard layout generator
ruifigueira Aug 9, 2023
bc05904
fix(docs): fix brazil keyboard layout hyperlink
ruifigueira Aug 17, 2023
10b3fc0
fix(input): skip deadkey if pressed key has no accented symbol
ruifigueira Aug 28, 2023
480e4ee
test(input): refactor keyboard layout tests
ruifigueira Aug 28, 2023
ca4717a
chore(input): code review changes
ruifigueira Aug 29, 2023
5b4e540
fix(input): better handling deadkeys
ruifigueira Aug 29, 2023
756770b
chore(input): code review changes
ruifigueira Aug 29, 2023
94cbdd5
test(input): convert multiple tests per layout into single test with …
ruifigueira Sep 19, 2023
c741f9d
docs(input): update version to v1.39
ruifigueira Sep 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/src/api/class-androiddevice.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,9 @@ Optional package name to launch instead of default Chrome for Android.
### option: AndroidDevice.launchBrowser.args = %%-browser-option-args-%%
* since: v1.29

### option: AndroidDevice.launchBrowser.keyboardLayout = %%-context-option-keyboard-layout-%%
* since: v1.**

## async method: AndroidDevice.longTap
* since: v1.9

Expand Down
6 changes: 6 additions & 0 deletions docs/src/api/class-browser.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,9 @@ await browser.CloseAsync();
### option: Browser.newContext.storageStatePath = %%-csharp-java-context-option-storage-state-path-%%
* since: v1.9

### option: Browser.newContext.keyboardLayout = %%-context-option-keyboard-layout-%%
* since: v1.**

## async method: Browser.newPage
* since: v1.8
- returns: <[Page]>
Expand All @@ -284,6 +287,9 @@ testing frameworks should explicitly create [`method: Browser.newContext`] follo
### option: Browser.newPage.storageStatePath = %%-csharp-java-context-option-storage-state-path-%%
* since: v1.9

### option: Browser.newPage.keyboardLayout = %%-context-option-keyboard-layout-%%
* since: v1.**

## async method: Browser.startTracing
* since: v1.11
* langs: java, js, python
Expand Down
3 changes: 3 additions & 0 deletions docs/src/api/class-browsertype.md
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,9 @@ use a temporary directory instead.
### option: BrowserType.launchPersistentContext.-inline- = %%-shared-context-params-list-v1.8-%%
* since: v1.8

### option: BrowserType.launchPersistentContext.keyboardLayout = %%-context-option-keyboard-layout-%%
* since: v1.**

## async method: BrowserType.launchServer
* since: v1.8
* langs: js
Expand Down
3 changes: 3 additions & 0 deletions docs/src/api/class-electron.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,3 +145,6 @@ Maximum time in milliseconds to wait for the application to start. Defaults to `

### option: Electron.launch.tracesDir = %%-browser-option-tracesdir-%%
* since: v1.36

### option: Electron.launch.keyboardLayout = %%-context-option-keyboard-layout-%%
* since: v1.**
11 changes: 11 additions & 0 deletions docs/src/api/class-keyboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -374,3 +374,14 @@ Dispatches a `keyup` event.
- `key` <[string]>

Name of the key to press or a character to generate, such as `ArrowLeft` or `a`.

## async method: Keyboard.changeLayout
* since: v1.**
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved

Changes keyboard layout.
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved

### param: Keyboard.changeLayout.layoutName
* since: v1.**
- `layoutName` <[string]>

%%-template-keyboard-layouts-%%
24 changes: 24 additions & 0 deletions docs/src/api/params.md
Original file line number Diff line number Diff line change
Expand Up @@ -734,6 +734,10 @@ Whether to allow sites to register Service workers. Defaults to `'allow'`.
* `'allow'`: [Service Workers](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API) can be registered.
* `'block'`: Playwright will block all registration of Service Workers.

## context-option-keyboard-layout
- `keyboardLayout` <[string]>

%%-template-keyboard-layouts-%%

## select-options-values
* langs: java, js, csharp
Expand Down Expand Up @@ -1731,3 +1735,23 @@ In this config:
1. Since `snapshotPathTemplate` resolves to relative path, it will be resolved relative to `configDir`.
1. Forward slashes `"/"` can be used as path separators on any platform.

## template-keyboard-layouts

Keyboard layout code. Currently, the following values are supported:

| Values | Name |
| :- | :- |
| `us`, `en-US` | [US English](https://learn.microsoft.com/en-us/globalization/keyboards/kbdus_7) <!-- 00000409 --> |
| `gb`, `en-GB` | [British](https://learn.microsoft.com/en-us/globalization/keyboards/kbduk) <!-- 00000809 --> |
| `dk`, `da-DK` | [Danish](https://learn.microsoft.com/en-us/globalization/keyboards/kbdda) <!-- 00000406 --> |
| `fr`, `fr-FR` | [French](https://learn.microsoft.com/en-us/globalization/keyboards/kbdfr) <!-- 0000040C --> |
| `de`, `de-DE` | [German](https://learn.microsoft.com/en-us/globalization/keyboards/kbdgr) <!-- 00000407 --> |
| `it`, `it-IT` | [Italian](https://learn.microsoft.com/en-us/globalization/keyboards/kbdit) <!-- 00000410 --> |
| `pt`, `pt-PT` | [Portuguese (Portugal)](https://learn.microsoft.com/en-us/globalization/keyboards/kbdpo) <!-- 00000816 --> |
| `br`, `pt-BR` | [Portuguese (Brazil)](https://learn.microsoft.com/en-us/globalization/keyboards/kbdbr_1) <!-- 00000416 --> |
| `ru`, `ru-RU` | [Russian](https://learn.microsoft.com/en-us/globalization/keyboards/kbdru) <!-- 00000419 --> |
| `ua`, `uk-UA` | [Ukrainian](https://learn.microsoft.com/en-us/globalization/keyboards/kbdur1) <!-- 00020422 --> |
| `es`, `es-ES`, `ca-ES` | [Spanish & Catalan (Spain)](https://learn.microsoft.com/en-us/globalization/keyboards/kbdsp) <!-- 0000040A --> |
| `latam` | [Spanish (Latin America)](https://learn.microsoft.com/en-us/globalization/keyboards/kbdla) <!-- 0000080A --> |
| `ch`, `de-CH` | [German (Switzerland)](https://learn.microsoft.com/en-us/globalization/keyboards/kbdsg) <!-- 00000807 --> |
| `fr-CH`, `it-CH` | [French and Italian (Switzerland)](https://learn.microsoft.com/en-us/globalization/keyboards/kbdsf_2) <!-- 0000100C --> |
4 changes: 4 additions & 0 deletions packages/playwright-core/src/client/input.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ export class Keyboard implements api.Keyboard {
async press(key: string, options: channels.PageKeyboardPressOptions = {}) {
await this._page._channel.keyboardPress({ key, ...options });
}

async changeLayout(layoutName: string): Promise<void> {
await this._page._channel.keyboardChangeLayout({ layoutName });
}
}

export class Mouse implements api.Mouse {
Expand Down
9 changes: 9 additions & 0 deletions packages/playwright-core/src/protocol/validator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -568,6 +568,7 @@ scheme.BrowserTypeLaunchPersistentContextParams = tObject({
recordHar: tOptional(tType('RecordHarOptions')),
strictSelectors: tOptional(tBoolean),
serviceWorkers: tOptional(tEnum(['allow', 'block'])),
keyboardLayout: tOptional(tString),
userDataDir: tString,
slowMo: tOptional(tNumber),
});
Expand Down Expand Up @@ -644,6 +645,7 @@ scheme.BrowserNewContextParams = tObject({
recordHar: tOptional(tType('RecordHarOptions')),
strictSelectors: tOptional(tBoolean),
serviceWorkers: tOptional(tEnum(['allow', 'block'])),
keyboardLayout: tOptional(tString),
proxy: tOptional(tObject({
server: tString,
bypass: tOptional(tString),
Expand Down Expand Up @@ -705,6 +707,7 @@ scheme.BrowserNewContextForReuseParams = tObject({
recordHar: tOptional(tType('RecordHarOptions')),
strictSelectors: tOptional(tBoolean),
serviceWorkers: tOptional(tEnum(['allow', 'block'])),
keyboardLayout: tOptional(tString),
proxy: tOptional(tObject({
server: tString,
bypass: tOptional(tString),
Expand Down Expand Up @@ -1119,6 +1122,10 @@ scheme.PageKeyboardPressParams = tObject({
delay: tOptional(tNumber),
});
scheme.PageKeyboardPressResult = tOptional(tObject({}));
scheme.PageKeyboardChangeLayoutParams = tObject({
layoutName: tString,
});
scheme.PageKeyboardChangeLayoutResult = tOptional(tObject({}));
scheme.PageMouseMoveParams = tObject({
x: tNumber,
y: tNumber,
Expand Down Expand Up @@ -2243,6 +2250,7 @@ scheme.ElectronLaunchParams = tObject({
strictSelectors: tOptional(tBoolean),
timezoneId: tOptional(tString),
tracesDir: tOptional(tString),
keyboardLayout: tOptional(tString),
});
scheme.ElectronLaunchResult = tObject({
electronApplication: tChannel(['ElectronApplication']),
Expand Down Expand Up @@ -2457,6 +2465,7 @@ scheme.AndroidDeviceLaunchBrowserParams = tObject({
recordHar: tOptional(tType('RecordHarOptions')),
strictSelectors: tOptional(tBoolean),
serviceWorkers: tOptional(tEnum(['allow', 'block'])),
keyboardLayout: tOptional(tString),
pkg: tOptional(tString),
args: tOptional(tArray(tString)),
proxy: tOptional(tObject({
Expand Down
3 changes: 3 additions & 0 deletions packages/playwright-core/src/server/DEPS.list
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,6 @@
./electron/
./firefox/
./webkit/

[input.ts]
./keyboards/
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,10 @@ export class PageDispatcher extends Dispatcher<Page, channels.PageChannel, Brows
await this._page.keyboard.press(params.key, params);
}

async keyboardChangeLayout(params: channels.PageKeyboardChangeLayoutParams, metadata?: CallMetadata): Promise<void> {
this._page.keyboard.changeLayout(params.layoutName);
}

async mouseMove(params: channels.PageMouseMoveParams, metadata: CallMetadata): Promise<void> {
await this._page.mouse.move(params.x, params.y, params);
}
Expand Down
124 changes: 106 additions & 18 deletions packages/playwright-core/src/server/input.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,12 @@
*/

import { assert } from '../utils';
import * as keyboardLayout from './usKeyboardLayout';
import type * as types from './types';
import type { Page } from './page';
import type { KeyboardLayout } from './keyboards';
import { defaultKeyboardLayout, defaultKeyboardLayoutName, keyboardLayoutNamesMapping } from './keyboards';

export const keypadLocation = keyboardLayout.keypadLocation;
export { keypadLocation } from './keyboards';

type KeyDescription = {
keyCode: number,
Expand All @@ -29,8 +30,13 @@ type KeyDescription = {
code: string,
location: number,
shifted?: KeyDescription;
deadKeyMappings?: Map<string, string>;
};

// either the key description or a sequence of keys for accented keys
// e.g. in portuguese, 'à' will be ['Shift+BracketRight', 'a']
type KeyboardLayoutClosure = Map<string, KeyDescription | string[]>;

const kModifiers: types.KeyboardModifier[] = ['Alt', 'Control', 'Meta', 'Shift'];

export interface RawKeyboard {
Expand All @@ -44,40 +50,66 @@ export class Keyboard {
private _pressedKeys = new Set<string>();
private _raw: RawKeyboard;
private _page: Page;
private _layoutName: string;
private _keyboardLayout: KeyboardLayoutClosure;
private _deadKeyMappings?: Map<string, string>;

constructor(raw: RawKeyboard, page: Page) {
this._raw = raw;
this._page = page;
const layoutName = page._browserContext._options.keyboardLayout;
this._keyboardLayout = getByKeyboardLayoutName(layoutName);
this._layoutName = layoutName ?? 'us';
}

changeLayout(layoutName: string) {
this._keyboardLayout = getByKeyboardLayoutName(layoutName);
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved
this._layoutName = layoutName;
this._deadKeyMappings = undefined;
}

async down(key: string) {
const description = this._keyDescriptionForString(key);
if (description.key !== 'Shift') this._deadKeyMappings = undefined;
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved
const autoRepeat = this._pressedKeys.has(description.code);
this._pressedKeys.add(description.code);
if (kModifiers.includes(description.key as types.KeyboardModifier))
this._pressedModifiers.add(description.key as types.KeyboardModifier);
const text = description.text;
await this._raw.keydown(this._pressedModifiers, description.code, description.keyCode, description.keyCodeWithoutLocation, description.key, description.location, autoRepeat, text);
await this._raw.keydown(this._pressedModifiers, description.code, description.keyCode, description.keyCodeWithoutLocation, description.key, description.location, autoRepeat, description.text);
}

private _keyDescriptionForString(keyString: string): KeyDescription {
let description = usKeyboardLayout.get(keyString);
let description = this._keyboardLayout.get(keyString);
assert(description, `Unknown key: "${keyString}"`);
assert(!Array.isArray(description), `Accented key "${keyString}" cannot be typed with a single keypress on "${normalizeLayoutName(this._layoutName)}" keyboard layout.\nUse type() method that will generate multiple key presses.`);

const shift = this._pressedModifiers.has('Shift');
description = shift && description.shifted ? description.shifted : description;

// if any modifiers besides shift are pressed, no text should be sent
if (this._pressedModifiers.size > 1 || (!this._pressedModifiers.has('Shift') && this._pressedModifiers.size === 1))
return { ...description, text: '' };
return description;

if (!this._deadKeyMappings || description.key === 'Shift') return description;
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved

// handle deadkeys / accented keys
const deadKeyText = this._deadKeyMappings.get(description.text);

// this will be handled properly in the future with IME support.
// For now, if there's no dead key mapping for current key, we'll skip it.
if (!deadKeyText) return description;

return { ...description, text: deadKeyText, key: deadKeyText };
}

async up(key: string) {
const description = this._keyDescriptionForString(key);
if (kModifiers.includes(description.key as types.KeyboardModifier))
this._pressedModifiers.delete(description.key as types.KeyboardModifier);
this._pressedKeys.delete(description.code);
await this._raw.keyup(this._pressedModifiers, description.code, description.keyCode, description.keyCodeWithoutLocation, description.key, description.location);
const descKey = description.deadKeyMappings ? 'Dead' : description.key;
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved
await this._raw.keyup(this._pressedModifiers, description.code, description.keyCode, description.keyCodeWithoutLocation, descKey, description.location);
if (description.key !== 'Shift') this._deadKeyMappings = description.deadKeyMappings;
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved
}

async insertText(text: string) {
Expand All @@ -87,8 +119,14 @@ export class Keyboard {
async type(text: string, options?: { delay?: number }) {
const delay = (options && options.delay) || undefined;
for (const char of text) {
if (usKeyboardLayout.has(char)) {
await this.press(char, { delay });
const descOrKeySequence = this._keyboardLayout.get(char);
if (descOrKeySequence) {
if (Array.isArray(descOrKeySequence)) {
for (const key of descOrKeySequence)
await this.press(key, { delay });
} else {
await this.press(char, { delay });
}
} else {
if (delay)
await new Promise(f => setTimeout(f, delay));
Expand Down Expand Up @@ -241,10 +279,38 @@ const aliases = new Map<string, string[]>([
['Enter', ['\n', '\r']],
]);

const usKeyboardLayout = buildLayoutClosure(keyboardLayout.USKeyboardLayout);
const defaultKeyboard = _buildLayoutClosure(defaultKeyboardLayout);
const cache = new Map<string, KeyboardLayoutClosure>(
// initialized with the default keyboard layout
[[defaultKeyboardLayoutName, defaultKeyboard]]
);

function getByKeyboardLayoutName(layoutName?: string): KeyboardLayoutClosure {
if (!layoutName) return defaultKeyboard;

const normalizedLayoutName = normalizeLayoutName(layoutName);
const klid = keyboardLayoutNamesMapping.get(normalizedLayoutName);
if (!klid) throw new Error(`Keyboard layout name "${layoutName}" not found`);

const cached = cache.get(klid);
if (cached) return cached;

const layout: KeyboardLayout = require(`./keyboards/layouts/${klid}`).default;
assert(layout, `No layout found for klid ${klid}`);
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved

function buildLayoutClosure(layout: keyboardLayout.KeyboardLayout): Map<string, KeyDescription> {
const result = new Map<string, KeyDescription>();
const result = _buildLayoutClosure(layout);

cache.set(klid, result);
return result;
}

function normalizeLayoutName(layoutName: string) {
return layoutName.replace(/-/g, '_').toLowerCase();
}

function _buildLayoutClosure(layout: KeyboardLayout): KeyboardLayoutClosure {
const result = new Map<string, KeyDescription | string[]>();
const accents = new Map<string, string[]>();
for (const code in layout) {
const definition = layout[code];
const description: KeyDescription = {
Expand All @@ -254,19 +320,22 @@ function buildLayoutClosure(layout: keyboardLayout.KeyboardLayout): Map<string,
code,
text: definition.text || '',
location: definition.location || 0,
deadKeyMappings: definition.key && definition.deadKeyMappings ? new Map(Object.entries(definition.deadKeyMappings)) : undefined,
};
if (definition.key.length === 1)
if (definition.key?.length === 1)
description.text = description.key;

// Generate shifted definition.
let shiftedDescription: KeyDescription | undefined;
if (definition.shiftKey) {
assert(definition.shiftKey.length === 1);
assert(definition.shiftKey === 'Dead' || definition.shiftKey.length === 1);
shiftedDescription = { ...description };
shiftedDescription.key = definition.shiftKey;
shiftedDescription.text = definition.shiftKey;
shiftedDescription.text = definition.shiftKey === 'Dead' ? '' : definition.shiftKey;
if (definition.shiftKeyCode)
shiftedDescription.keyCode = definition.shiftKeyCode;
if (definition.shiftDeadKeyMappings)
shiftedDescription.deadKeyMappings = new Map(Object.entries(definition.shiftDeadKeyMappings));
}

// Map from code: Digit3 -> { ... descrption, shifted }
Expand All @@ -286,10 +355,29 @@ function buildLayoutClosure(layout: keyboardLayout.KeyboardLayout): Map<string,
if (description.key.length === 1)
result.set(description.key, description);

// Map from shiftKey, no shifted
if (shiftedDescription)
result.set(shiftedDescription.key, { ...shiftedDescription, shifted: undefined });
// Map from accented keys
if (definition.deadKeyMappings) {
for (const [k, v] of Object.entries(definition.deadKeyMappings))
accents.set(v, [code, k]);
}

if (shiftedDescription) {
if (shiftedDescription.key !== 'Dead')
// Map from shiftKey, no shifted
result.set(shiftedDescription.key, { ...shiftedDescription, shifted: undefined });

// Map from shifted accented keys
if (definition.shiftDeadKeyMappings) {
ruifigueira marked this conversation as resolved.
Show resolved Hide resolved
for (const [k, v] of Object.entries(definition.shiftDeadKeyMappings))
accents.set(v, [`Shift+${code}`, k]);
}
}
}

for (const [k, v] of accents.entries())
// if result already has a dedicated accented key, we don't add deadkey generated accents
if (!result.has(k)) result.set(k, v);

return result;
}

Expand Down
2 changes: 2 additions & 0 deletions packages/playwright-core/src/server/keyboards/DEPS.list
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[*]
./layouts/
Loading