At what point is it acceptable to use new syntax or interfaces?

[gibson042]

(prompted by #3192 (comment))

To what extent are non-original or "new" features appropriate in testing unrelated older functionality? For example, should an intentionally limited implementation like Hermes or QuickJS or XS or a new in-progress implementation be unable to even run tests covering ES6 functionality because it does not yet support class private fields or String.prototype.matchAll?

Ideally, this is resolved by updating CONTRIBUTING.md with appropriate guidance.

[ljharb]

My hope would be that tests can run in the oldest possible engine, and don't assume new features, because then test262 tests can also be run against polyfills in those older engines.

However, I believe historically this hasn't been a priority for test authoring.

[leobalter]

There is an abstract line of what is old enough, I feel it's subjective and I'd say things dating from the last 2 years are just fine?

The other workaround is flagging things using the features metadata.

There is a lot of maintenance required for targeting multiple versions of EcmaScript, and this also means how we deal with incoming PRs.

Relieving the rules here will result in avoiding asking PR authors to modify their code to not use modern ES syntax/api. I agree we should avoid edgy features whenever necessary but some are still ok, IMO, e.g. object method definitions, symbols, async functions, optional catch binding.

[ljharb]

If the intention is to be able to test polyfills, "time" is insufficient; it'd be "whatever polyfills need to run in", which for all of mine includes IE 6 (ie, ES3 syntax).

[leobalter]

Some rationale: It's rare to have bug fixes for tests of features released in ES more than 1 or 2 years. Way more often we have small spec changes, such as PRs to ECMA-262, changing bits of the normative text and adapting tests here.

The compatibility for older ES versions is pretty much compromised, because it flows between what was set in the older ES version to features of that older version that had their adaptation. I wonder if some/most of these implementations are just shiping modern ES but using a subset of the current features. This is likely what happens with XS opting out of Intl API or cross realms functionality.

It's pretty complex to have all the subset implementations and coordinate with the PRs volunteers to make everything compatible. Requiring contributors to use old features only creates a burden in maintenance.

[jugglinmike]

Thanks for raising this, @gibson042. You're not the first to wonder; we really need to nail down a policy.

Although I've always tried to avoid post-ES5 features whenever possible, I'm only recently wondering if transpilers should inform the policy. In recent years, a huge number of free tools have been written that can rewrite specific aspects of source code. Maybe the more aggressive of the restrictions that we're considering imposing on contributors could be obviated by a recommendation to consumers to use those tools according to their needs.

@gibson042 @ljharb (and anyone else who has an interest in running Test262 tests in lagging engines): could you use a tool like Babel to rewrite superfluous features which aren't available in your environment?

[ljharb]

No, because then i have to rely on the transpiler's output of the tests being accurate, and that's not always a guarantee. JS can't be reliably and consistently transpiled 1:1 (only a small subset of features can - even arrow functions come with caveats)

[gibson042]

@linusg articulates a useful position at #3845 (comment)

The primary reason why I want to keep core harness files as simple as possible is to shrink the language subset required to start running test262.

[ljharb]

My motivation for #3196 (comment) is that, and also to shrink the subset required to run tests for anything polyfillable.

[ptomato]

I'm generally in agreement with the above, but I'd prefer not to be strict about this: I'd rather err on the side of accepting contributions that use newer syntax, and then also cheerfully and promptly accept further contributions such as #3845 from test consumers who run into problems with newer syntax.

We could also consider letting people use newer syntax when developing tests (such as Temporal, where saving as much development time as possible per test has been really helpful due to the size of the proposal) and then transpiling them once later.

[gibson042]

Related (specifically w.r.t. harness code): #3032