Introducing browser.i18n.getOSLanguage

[carlosjeurissen]

Motivation

As browsers often don't directly use the OS language but some reflection of it based on what languages are supported by the browser UI. Having the original OS language is useful for language-related extensions and for extensions who want to provide translations more true to the OS language (like be able to adapt to a specific sub-language, like Belgium-Dutch (nl-BE). While i18n.getUILanguage would return 'nl'.

Syntax

i18n.getOSLanguage() would synchronously return an IETF tag just like i18n.getUILanguage() does right now.

It would follow the following signature for i18n.json:

{
  "name": "getOSLanguage",
  "type": "function",
  "nocompile": true,
  "description": "Gets the UI language of the Operating System. This is different from $(ref:i18n.getUILanguage) which returns the UI language of the web browser.",
  "parameters": [],
  "returns": {
    "type": "string",
    "description": "An IETF BCP 47 language tag such as en-US or pt-BR."
  }
},

See proposal: #569

[xeenon]

This sounds fine to me for Safari.

[hanguokai]

Why not use navigator.language and navigator.languages?

[xeenon]

Why not use navigator.language and navigator.languages?

It isn't available in service workers, IIRC.

[birtles]

Why not use navigator.language and navigator.languages?

It isn't available in service workers, IIRC.

It seems like it should be available given ServiceWorkerGlobalScope inherits from WorkerGlobalScope whose navigator member has type WorkerNavigator which includes NavigatorLanguage.

That said, I believe navigator.language / navigator.languages is a separate concern to this.

As I understand it we have:

1. The language of the OS (browser.i18n.getOSLanguage, i.e. this proposal)
2. The language of the browser chrome (browser.i18n.getUILanguage)
3. The languages the user has indicated they prefer to receive content in (navigator.language / navigator.languages)

So (3) is not really a replacement for (1), or (2) for that matter.

I'm a little bit concerned that adding (1) might lead to a worse user experience when add-on authors don't appreciate the difference between (1) and (2).

For example, a lot of users are on a non-English OS but deliberately install an English browser, or vice-versa. I'm afraid if extension authors reach for browser.i18n.getOSLanguage by default, then users will often experience a case where their extension popups etc. are in a different language to the rest of the browser chrome.

[hanguokai]

@xeenon navigator.language(s) is available in service worker on all browsers. You can open a service worker console to test it.

Because navigator.language(s) represents the user's preferred language(s), which is the same concept as OS language preference, I don't think that browser.i18n.getOSLanguage or navigator.OSlanguage(s) is needed.

By default, browsers uses the language settings of the operating system. And users can change it in browser settings independently.

[birtles]

Because navigator.language(s) represents the user's preferred language(s), which is the same concept as OS language preference, I don't think that browser.i18n.getOSLanguage or navigator.OSlanguage(s) is needed.

I think browser.i18n.getUILanguage is still helpful since I think you want chrome generated by add-ons to track the language of the browser chrome.

It's quite common to have the accept languages (navigator.language(s)) of the browser differ from the language of the browser chrome itself.

For example, I need to set navigator.language to ja in order to login to the Japanese corporate e-tax site but I need to set it to en-* in order to login to an Australian bank. I don't think chrome generated by add-ons should track that since the browser chrome doesn't.

[hanguokai]

For example, I need to set navigator.language to ja in order to login to the Japanese corporate e-tax site but I need to set it to en-* in order to login to an Australian bank.

This is a common i18n issue. Websites that supports multiple languages usually have a menu for users to change website's language. The priority order of deciding the language is as follows:

1. url parameter (?lang=ja, /en-us/path/).
2. cookie (save user's setting by website's language menu).
3. navigator.language(Accept-Language header) by default

It's quite common to have the accept languages (navigator.language(s)) of the browser differ from the language of the browser chrome itself.

Yes, this is why there is a browser.i18n.getUILanguage. But the original topic talks about browser.i18n.getOSLanguage.

Browser extensions can also supply a language-select menu for users in the extension's settings if it supports multiple languages. Note: currently chrome.i18n.getMessage doesn't support return a specified language message(crbug/660704), but developers have workarounds.

[birtles]

Yes, I think we're on the same page here -- I agree that browser.i18n.getOSLanguage might not be necessary.

(And I also agree that i18n.getMessage() should allow specifying a different language and have often had users request that behavior for one of my extensions.)

[carlosjeurissen]

@hanguokai navigator.languages / navigator.language (the accept-language header), allow a user to tell websites what languages they understand / want the websites they visit to be in. This is very different from the language of the OS. The OS language does not even have to be present in navigator.languages.

As I was trying to state in the first post, the OS language is different from the browser UI language. For example, windows supports 87 languages while Google Chrome only supports 54 languages. And there are is always 1:1 maping. For example, Google Chrome doesn't support Galician. Instead it probably uses Portuguese.

However, as extension you want to know the original language before it gets mapped to the language a browser supports. One additional use-case on top of the already named ones is the ability to use exactly the same language between an extension and their native companion app.

@hanguokai @birtles The ability to get a specified message in a specified language using i18n.getMessage would be very welcome. I propose we make a dedicated issue for this. I've briefly proposed something like this in the past, see: #93 (comment). One potential issue if we want to keep the API synchronous is performance. Yet we can discuss this in a separate issue.

[birtles]

However, as extension you want to know the original language before it gets mapped to the language a browser supports. One additional use-case on top of the already named ones is the ability to use exactly the same language between an extension and their native companion app.

Gotcha, that makes sense.

For what browsers does such a mapping exist? In the case of something like Firefox, the browser localization is simply whatever language the user chose to download. In that situation, it's hard to determine which language the user actually prefers.

My concern is that extension developers, when presented with getUILanguage() and getOSLanguage(), may not know which one to choose and many might simply see getOSLanguage() and use that but for 99% of the cases, they really should use getUILanguage() to be consistent with the browser's chrome. So perhaps a flag to getUILanguage, e.g. getUILanguage({ basis: 'os' }) would make it harder to make that mistake?

[hanguokai]

the OS language may be different from the browser UI language.

This is not an extension-specific issue. All websites also face this problem. I think this should be discussed in a more general group in W3C.

For example, windows supports 87 languages while Google Chrome only supports 54 languages.

Chrome UI maybe only support 54 languages. But UI languages does not equal navigator.languages. I just checked browser's languages settings:

• Chrome: almost 160 languages in chrome://settings/languages
• Firefox: 100+ languages in about:preferences -> Language

In order to choose the best language, extensions can check both navigator.language and browser.i18n.getUILanguage. If they are different, it's hard to say which is better. But at least users can understand them.

[carlosjeurissen]

@birtles When it comes to Firefox, the user can actually change the language of the browser UI in the Firefox settings. So this even increases the use-case for i18n.getOSLanguage, as it can't always be be approached with i18n.getUILanguage.

As for the confusion between i18n.getOSLanguage and i18n.getUILanguage, we are dealing with developers here. It can be a certain percentage but I don't think it will be 99%. The parameter passed in getUILanguage seems quite interesting. However, it can not be feature detected (unlike 'getOSLanguage' in browser.i18n) . Thus problematic for backwards compatibility. In addition, passing an object to getUILanguage currently raises an exception.

Thus it seems best to stick with i18n.getOSLanguage. Showcasing the difference between getOSLanguage and getUILanguage can be done in the documentation. Having good documentation is already a must as getUILanguage by default has wider browser support at this moment of time.

@hanguokai Making the OS language available to ordinary websites has different considerations and will take a lot more time and my personal estimate is it will not make it. Implementing it for browser extensions first seems like the right path to me. Just like we did with i18n.getAcceptLanguages, which later was made available using navigator.languages.

[hanguokai]

As for the confusion between i18n.getOSLanguage and i18n.getUILanguage, we are dealing with developers here.

Counting i18n.getOSLanguage, how do developers choose from i18n.getOSLanguage, i18n.getUILanguage and navigator.language? If they're different. I'd like to hear from other developers.

Developers may not be able to make choices for users, so I know some developers offer language options to users (not only extension developers face this problem, desktop and mobile app developers also support this way). This allows the user to choose a language they prefer, which may be neither i18n.getOSLanguage nor i18n.getUILanguage. For example, my OS and Browser language is en-US, but I would like to choose zh-CN in my extensions.

How to integrate i18n.getOSLanguage with i18n.getMessage()? For example, i18n.getMessage(key, {prefer: 'OS' or 'Browser'}). But in this case, I would like to let the user decide which language they prefers. So I prefer i18n.getMessage(key, {lang: 'zh-CN}) described at #258.

[hanguokai]

How to integrate i18n.getOSLanguage with i18n.getMessage()?

In addition, there is another way to use i18n messages in manifest.json and CSS files, __MSG_messagename__. If users prefer OS language, these messages should also use OS language.

[carlosjeurissen]

Counting i18n.getOSLanguage, how do developers choose from i18n.getOSLanguage, i18n.getUILanguage and navigator.language? If they're different. I'd like to hear from other developers.

They simply use the one which works best in their situation. getOSLanguage to get the OS language, getUILanguage to get the language of the browser UI, and navigator.languages to get the acceptLanguages. navigator.language is unreliable and should thus not be used.

Developers may not be able to make choices for users, so I know some developers offer language options to users (not only extension developers face this problem, desktop and mobile app developers also support this way). This allows the user to choose a language they prefer, which may be neither i18n.getOSLanguage nor i18n.getUILanguage. For example, my OS and Browser language is en-US, but I would like to choose zh-CN in my extensions.

It is not about developers making choices for users. If users prefer choice, developers can use the setCurrentLanguage proposal as mentioned here: #258. The i18n.getOSLanguage proposal is not at all competing with the i18n.setCurrentLanguage proposal.

How to integrate i18n.getOSLanguage with i18n.getMessage()? For example, i18n.getMessage(key, {prefer: 'OS' or 'Browser'}). But in this case, I would like to let the user decide which language they prefers. So I prefer i18n.getMessage(key, {lang: 'zh-CN}) described at #258.

See your own proposal, #258

In addition, there is another way to use i18n messages in manifest.json and CSS files, __MSG_messagename__. If users prefer OS language, these messages should also use OS language.

Why should they be? If the complete extension language should be changed, this should be done with #258. In there, we can discuss whether or not these __MSG_messagename__ should also change depending on the set language.

[Rob--W]

The API proposal suggests one return value. Should we suggest an array of values instead, in case a user has configured multiple languages?

I don't have a strong preference either way.

[carlosjeurissen]

@Rob--W Good point. This would also solve any potential confusion raised in by @birtles in #252 (comment) with i18n.getUILanguage.

@xeenon @rdcronin Do you have any opposition in changing this to browser.i18n.getOSLanguages? If not, I will go ahead and update the proposal.

[hanguokai]

Should we suggest an array of values instead, in case a user has configured multiple languages?

Some operating systems do allow setting up multiple languages, the first of which is used as the OS UI language, like macOS and Android. It is similar to i18n.getAcceptLanguages() or navigator.languages. They are used for the fallback mechanism at OS level.

What results are returned depends on what the API is actually used for.

• If you want to compare which language is more precise, then returning one is enough. For example, i18n.getUILanguage() returns nl, but i18n.getOSLanguage() returns nl-BE.
• If you want to implement a better fallback mechanism than the i18n.getMessage() built-in mechanism and want to support as many languages as possible, then returning an array is better.

[xeenon]

Do you have any opposition in changing this to browser.i18n.getOSLanguages? If not, I will go ahead and update the proposal.

@carlosjeurissen I'm not sure. Now that I look at it, I think In out implementation that will effectively be the same result as i18n.getAcceptLanguages(). In fact, i18n.getOSLanguage() will likely just be the first result of that array in Safari / WebKit.

[carlosjeurissen]

@xeenon From what I understand it seems Safari derives it's Accept-Languages header from the languages selected in System Settings. However, it does not seem to be a 1:1. Like other browsers do for Accept-Languages it adds other more general language tags in between.

Say en-GB and nl-BE are selected in system preferences. Your Accept-Languages will look like en-GB,en,nl-BE,nl.

i18n.getOSLanguages or i18n.getSystemLanguages would ideally omit this and provide the extension with the way users configured it in their settings.

[carlosjeurissen]

@hanguokai Thanks for your thoughts shared in #569 (comment) To keep the discussion in one thread, replying in this issue.

You are right about the difference between the OS preferred languages and the language the OS actually picks. It is not always the first language. In macOS, when selecting "Georgian" as primary language, it will skip it in search for a language the OS actually supports.

This means we can not simply assume the first language returned by i18n.getOSLanguages is the same as the language of the OS. Yet it seems this is what Firefox currently does on macOS.

Considering this having two different methods, i18n.getOSLanguage and i18n.getOSLanguages (or different naming) at the same time would be needed.

Using prefer: "browser" on the existing i18n.getUILanguage and i18n.getAcceptLanguages would not be backwards compatible. So I would be inclined not to use this pattern. Edit: as @hanguokai pointed out in #252 (comment), it would be backwards compatible as long as you skip passing the prefer: "browser" when requesting the "browser" values.

@Rob--W in reply to your comment #569 (comment)
For i18n.getOSLanguages we would indeed want it to return a non-empty array. As for the discussion on i18n.getOSLanguage vs i18n.getOSLanguages, when checking the Firefox code, it seems not so straightforward to figure out the actual language the OS currently uses. i18n.getOSLanguages is easier to implement in that regard. I assume the same is true for other browsers.

[hanguokai]

Using prefer: "browser" on the existing i18n.getUILanguage and i18n.getAcceptLanguages would not be backwards compatible.

The new parameter is optional and has a default value, so it is backwards compatible.

Here the OS and the browser use the similar and overlapped concept (one UI language and a list of accept languages), IMO it makes sense that they share the same methods. Otherwise using four new methods as I described in #569 (comment) , whose names are symmetrical and clear, so developers will not confuse their functionality.

[carlosjeurissen]

@hanguokai Good point! Seems for getUILanguage we could add an option to either fetch the language of the OS or the Browser. As for getSystemLanguages, I am inclined to not link it to getAcceptLanguages as it seems different concepts. One is meant for websites and what content to return. While the System Languages is purely for language display.

@xeenon do you have an idea on how doable it is to find out what the language of the UI of the OS actually is? (versus the first system language, as this might not be true when your first system language is Georgian for example).

[xeenon]

@carlosjeurissen We have a method to get the single device (OS) locale on Apple platforms. So I would prefer this method stay as returning a single string, not an array.

Given an accept languages array of [ "bn-IN", "en-IN" ], the system returns en-GB. This occurs because bn (Bengali) lacks system localization, and en-IN (Indian English) falls back to its parent language, en-GB (British English), which is then used for all system apps and services' localization. The getOSLanguage function in our case would return en-GB.

[Rob--W]

@carlosjeurissen We have a method to get the single device (OS) locale on Apple platforms. So I would prefer this method stay as returning a single string, not an array.

That could also be turned into a single-value array.

It is however not possible to put multiple values in one string.

I don't feel very strongly about either approach, but from the API design POV, an array would make sense if there are systems with multiple values.

[xeenon]

That could also be turned into a single-value array.

Indeed! I'm fine with a single value array for flexibility.

[carlosjeurissen]

@xeenon @Rob--W Then I conclude ideally extensions would be offered two additional APIs.

One, i18n.getSystemLanguages which would return an Array

[ "bn-IN", "en-IN"]

And another which gives the language the os actually uses as a string. As far as I know this would only be one value. Tho I am fine with this returning an array with a single value just in case.

i18n.getOSLanguage which would give en-GB.

This way an extension can both match their native app as well as the language the OS actually uses. This is also helpful for a bunch of other usecases in which the user needs to be given certain instructions to be followed in the OS.

[xeenon]

@carlosjeurissen Why is one method system and the other OS? Should they both use the same term?

[carlosjeurissen]

@xeenon we can use OS or System for both. I do not feel very strongly about the naming as long as the functionality is there. To me "OS" feels more UI like while System feels more under the hood like.

@Rob--W how do you feel about naming?
@Rob--W @xeenon shall I proceed and update the PR to include both methods? Do we want i18n.getOSLanguage to return a single value or array with single value?

[xeenon]

I'm fine either way on i18n.getOSLanguage returning an array or a string, as long as the name reflects the plural or singular nature of the result.

[hanguokai]

Returning an array or a string ? As I said at #569 (comment) , we can support both, because they are different concepts. Personally, we can use the existing methods with a new optional parameter.

// return a single string for UI language
i18n.getUILanguage(
    prefer: "browser" // or "os"
);

// return an array for accept languages
i18n.getAcceptLanguages(
    prefer: "browser" // or "os"
);