Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Date.to(UTC)String() outputs match web reality and spec language #4067

Merged
merged 1 commit into from
Nov 22, 2017
Merged

Make Date.to(UTC)String() outputs match web reality and spec language #4067

merged 1 commit into from
Nov 22, 2017

Conversation

xiaoyinl
Copy link
Contributor

@xiaoyinl xiaoyinl commented Oct 25, 2017

Ref #4178

Sec 20.3.3.2 requires Date.parse should be able to parse the values returned by Date.toString(), Date.toUTCString() and Date.toISOString() in each engine's own implementation. In ChakraCore, if the year is <100, toString() and toUTCString() don't pad the year to 4 digits, and thus Date.parse can't parse them.

This PR makes two changes:

  1. If the absolute value of the year is less than 1000, pad it to 4 digits
  2. For a BC year, output negative year, rather than positive year + "B.C."

Copy link
Contributor

@dilijev dilijev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation and tests look good so far. Since this is a strict improvement over previously, I'd be willing to merge this change as-is, but if you'd like to continue work on the cases not yet handled, I can hold off. Let me know how you'd like to proceed.

@@ -404,7 +404,7 @@ namespace Js {
// Add the year.
if (pymd->year > 0)
{
bs->AppendChars(pymd->year, 10, ConvertLongToString);
bs->AppendChars(static_cast<uint32>(pymd->year), 10, ConvertUInt32ToString_ZeroPad_4);
Copy link
Contributor

@dilijev dilijev Nov 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did something change so that now this static_cast is necessary where it wasn't before?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember I put static_cast there only for stylistic reason (making int to uint32 conversion explicit). It's unnecessary so I removed it.

const errno_t err = _ultow_s(value, buffer, charCapacity, 10);
Assert(err == 0);
}
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: newline after this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general the above function is a bit ugly but I understand the need to implement ~ this way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't super clear from the read that the reason for converting the input number to uint16 is so that it is compatible with the char16 _u('0'). Can you add a comment explaining? What will behavior look like for years > 4 digits and > MAX_UINT16 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I fixed it by moving the static_cast to where the int32 value is assigned to char16, for readability and avoided > MAX_UINT16 issue.

@@ -418,7 +451,14 @@ namespace Js {
bs->AppendChars(_u(' '));

// year is directly after day, month, daydigit for IE11+
bs->AppendChars(pymd->year, 10, ConvertLongToString);
if (pymd->year > 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: wrap the pymd->year in parens for readability.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@dilijev dilijev self-assigned this Nov 8, 2017
@dilijev
Copy link
Contributor

dilijev commented Nov 8, 2017

Ah, I noticed the test code is assuming a complete implementation. If we take this as-is we should disable the failing test cases for now.

@dilijev
Copy link
Contributor

dilijev commented Nov 8, 2017

Also looks like the code doesn't yet handle negative years for the year padding:

Date.parse can't parse "Sat Nov 13 -1 17:26:37 GMT-0800 (Pacific Standard Time)", returned by toString() on Date -000001-11-14T01:26:37.343Z

(Note the year -000001)

@xiaoyinl
Copy link
Contributor Author

xiaoyinl commented Nov 8, 2017

@dilijev Thanks for the review! I prefer to merge the current code change in this PR, and I will open another PR for making Date.parse recognize zero-padded years. I will address your comments, and update the tests so that they only test the outputs of toString.

@dilijev
Copy link
Contributor

dilijev commented Nov 8, 2017

After this change, we would conform to the other engines in how we toString dates with positive years.

┌──────────────────┬───────────────────────────────────────────────────────────┐
│ ch-1.7.1         │ Fri Oct 12 1 22:16:33 GMT-0700 (Pacific Daylight Time)    │
│ ch-master-latest │ Wed Oct 12 11 22:16:33 GMT-0700 (Pacific Daylight Time)   │
│ node-ch          │ Mon Oct 12 111 22:16:33 GMT-0700 (Pacific Daylight Time)  │
│                  │ Thu Oct 12 1111 22:16:33 GMT-0700 (Pacific Daylight Time) │
├──────────────────┼───────────────────────────────────────────────────────────┤
│ d8               │ Fri Oct 12 0001 22:16:33 GMT-0700 (Pacific Daylight Time) │
│ jsc              │ Wed Oct 12 0011 22:16:33 GMT-0700 (Pacific Daylight Time) │
│ sm               │ Mon Oct 12 0111 22:16:33 GMT-0700 (Pacific Daylight Time) │
│                  │ Thu Oct 12 1111 22:16:33 GMT-0700 (Pacific Daylight Time) │
├──────────────────┼───────────────────────────────────────────────────────────┤
│ node             │ Fri Oct 12    1 22:16:33 GMT-0700 (Pacific Daylight Time) │
│                  │ Wed Oct 12   11 22:16:33 GMT-0700 (Pacific Daylight Time) │
│                  │ Mon Oct 12  111 22:16:33 GMT-0700 (Pacific Daylight Time) │
│                  │ Thu Oct 12 1111 22:16:33 GMT-0700 (Pacific Daylight Time) │
└──────────────────┴───────────────────────────────────────────────────────────┘

I wonder why node uses spaces instead of leading 0's on the year?
/cc @bterlson, @digitalinfinity: do you have any insight here?

@xiaoyinl xiaoyinl changed the title WIP: make Date.parse able to parse Date.toString() Date.to(UTC)String() should pad years to 4 digits Nov 8, 2017
@dilijev
Copy link
Contributor

dilijev commented Nov 8, 2017

Handling of negative years has less inter-engine agreement:

## Source
let year1BC = new Date("0001-10-13T05:16:33Z");
let year11BC = new Date("0001-10-13T05:16:33Z");
year1BC.setFullYear(-1, 10, 13);
year11BC.setFullYear(-11, 10, 13);

print(year1BC.toString());
print(year11BC.toString());


┌──────────────────┬────────────────────────────────────────────────────────────┐
│ d8               │ Sat Nov 13 -001 22:16:33 GMT-0800 (Pacific Standard Time)  │
│ jsc              │ Mon Nov 13 -011 22:16:33 GMT-0800 (Pacific Standard Time)  │
├──────────────────┼────────────────────────────────────────────────────────────┤
│ ch-1.7.1         │ Sat Nov 13 -1 22:16:33 GMT-0800 (Pacific Standard Time)    │
│ ch-master-latest │ Mon Nov 13 -11 22:16:33 GMT-0800 (Pacific Standard Time)   │
│ node-ch          │                                                            │
├──────────────────┼────────────────────────────────────────────────────────────┤
│ sm               │ Sat Nov 13 -0001 22:16:33 GMT-0800 (Pacific Standard Time) │
│                  │ Mon Nov 13 -0011 22:16:33 GMT-0800 (Pacific Standard Time) │
├──────────────────┼────────────────────────────────────────────────────────────┤
│ node             │ Sat Nov 13   -1 22:16:33 GMT-0800 (Pacific Standard Time)  │
│                  │ Mon Nov 13  -11 22:16:33 GMT-0800 (Pacific Standard Time)  │
└──────────────────┴────────────────────────────────────────────────────────────┘


// test BC years
let year1BC = new Date();
let year11BC = new Date();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be better to do new Date("0001-10-13T05:16:33Z") (with a specific initialization string) so that the values and output is more predictable.

@dilijev
Copy link
Contributor

dilijev commented Nov 8, 2017

Added bug #4178

@xiaoyinl
Copy link
Contributor Author

xiaoyinl commented Nov 9, 2017

@dilijev I have updated the code, and it now also pads negative years to 4 digits, matching SpiderMonkey, because I think this might make Date.parse() easier to parse these years.

@dilijev
Copy link
Contributor

dilijev commented Nov 9, 2017

@xiaoyinl Thanks -- also I happen to think that format makes the most sense. :)

@@ -385,28 +385,29 @@ namespace Js {
const auto ConvertUInt32ToString_ZeroPad_4 = [](const uint32 value, char16 *const buffer, const CharCount charCapacity)
{
Assert(charCapacity >= 4);
uint16 value_uint16 = static_cast<uint16>(value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I see the value here. Should be the same logic whether 32 or 16 bit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember I put it there, because otherwise I need to cast value + _u('0') to 16 bit every time it's assigned to buffer[], which takes char16.

const errno_t err = _ultow_s(value, buffer, charCapacity, 10);
Assert(err == 0);
}
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't super clear from the read that the reason for converting the input number to uint16 is so that it is compatible with the char16 _u('0'). Can you add a comment explaining? What will behavior look like for years > 4 digits and > MAX_UINT16 ?

else
{
bs->AppendChars(_u('-'));
bs->AppendChars(-(pymd->year), 10, ConvertUInt32ToString_ZeroPad_4);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than passing in a negated negative number, maybe use a local variable named to make the intent more clear? Like
positiveYear = -(pymd->year); // pymd->year is negative


runTest(new Date("0001-10-13T05:16:33Z"),
"Fri Oct 12 0001 22:16:33 GMT-0700 (Pacific Daylight Time)",
"Sat, 13 Oct 0001 05:16:33 GMT");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this test would read better if the definition of runTest were at the top of the file.

Also, you might consider using UnitTestFramework.js (see usage in other tests) and using asserts instead of rolling your own messages for failures.

// Negative years are also padded to four digits
runTest(new Date("-000001-10-13T05:16:33Z"),
"Tue Oct 12 -0001 22:16:33 GMT-0700 (Pacific Daylight Time)",
"Wed, 13 Oct 0002 B.C. 05:16:33 GMT");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why toUTCString not only prints GMT timezone (as opposed to UTC), but also switches from negative year to B.C. year. I think we should consult the spec and other engines on what the outputs should look like.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this date (-000001-10-13T05:16:33Z), both v8 and SpiderMonkey output GMT timezone and negative years: Wed, 13 Oct -001 05:16:33 GMT and Wed, 13 Oct -0001 05:16:33 GMT respectively.

@xiaoyinl
Copy link
Contributor Author

@dilijev Sorry for the delay! I was a little busy these days. I think I have addressed the issues you mentioned in the above comments. UnitTestFramework.js is used. It's indeed much more convenient.

@xiaoyinl
Copy link
Contributor Author

I just realized Section 20.3.4.43 specifies the exact format of toUTCString(). It says:

[...]
7. Let year be the String representation of YearFromTime(tv), formatted as a number of at least four digits, padded to the left with zeroes if necessary.
8. Return the string-concatenation of weekday, ",", the code unit 0x0020 (SPACE), month, the code unit 0x0020 (SPACE), day, the code unit 0x0020 (SPACE), year, the code unit 0x0020 (SPACE), and TimeString(tv).

So it seems toUTCString() should print a negative year, instead of a positive year + "B.C.".

Also I'm confused that based on item 8, it seems an UTCString should look like Mon, Nov 20 2017 15:16:33 GMT (month before day), but v8, SpiderMonkey and ChakraCore all print Mon, 20 Nov 2017 15:16:33 GMT (day before month). Do I misunderstand the spec or should we follow that?

/cc @bterlson

@dilijev
Copy link
Contributor

dilijev commented Nov 21, 2017

We generally go with the flow of 3/4 agreement (web reality). In this case looks like we all already agree, and the spec should be updated to match the web reality. I'll open a PR on the spec. /cc @bterlson

Printing B.C. definitely seems to be a bug.

## Source
let posYear = new Date('2008-10-13T05:16:33Z').toUTCString();
print(posYear);

┌──────────────────┬───────────────────────────────┐
│ ch-1.6-latest    │ Mon, 13 Oct 2008 05:16:33 GMT │
│ ch-1.7-latest    │                               │
│ ch-1.7.4         │                               │
│ ch-1.8-latest    │                               │
│ ch-master-latest │                               │
│ d8               │                               │
│ jsc              │                               │
│ node             │                               │
│ node-ch          │                               │
│ sm               │                               │
└──────────────────┴───────────────────────────────┘

## Source
let negYear = new Date((new Date('2008-10-13T05:16:33Z')).setFullYear(-2008, 10, 13)).toUTCString();
print(negYear);

┌──────────────────┬────────────────────────────────────┐
│ ch-1.6-latest    │ Sat, 14 Nov 2009 B.C. 06:16:33 GMT │
│ ch-1.7-latest    │                                    │
│ ch-1.7.4         │                                    │
│ ch-1.8-latest    │                                    │
│ ch-master-latest │                                    │
│ node-ch          │                                    │
├──────────────────┼────────────────────────────────────┤
│ d8               │ Sat, 14 Nov -2008 06:16:33 GMT     │
│ jsc              │                                    │
│ node             │                                    │
│ sm               │                                    │
└──────────────────┴────────────────────────────────────┘

@dilijev dilijev changed the base branch from master to release/1.8 November 21, 2017 02:07
@dilijev
Copy link
Contributor

dilijev commented Nov 21, 2017

@xiaoyinl I retargeted this PR to release/1.8. Would you mind cherry-picking or rebasing your changes on top of release/1.8 (to leave out the master-only changes that are not in release/1.8), and then force-pushing your branch?

@dilijev
Copy link
Contributor

dilijev commented Nov 21, 2017

@xiaoyinl Thanks!

@xiaoyinl
Copy link
Contributor Author

@dilijev I've cherry-picked on top of release/1.8 and I also changed the B.C. to printing negative years, so that it should now completely match SpiderMonkey.

@xiaoyinl xiaoyinl changed the title Date.to(UTC)String() should pad years to 4 digits Make Date.to(UTC)String() outputs match SpiderMonkey Nov 21, 2017
@dilijev
Copy link
Contributor

dilijev commented Nov 21, 2017

Hm... for the title, I think it would be better to say "Make Date.to(UTC)String() outputs match web reality and spec language".

Spec language [assumptions in brackets] re: negative years:

The [absolute value of the] year is padded to 4 digits as indicated in spec. [For negative numbers, it is then preceeded by a -, which is not discussed in the spec, and I'll make a clarifying PR to the spec to address that point.]

It's generally not our goal to match any particular other engine, regardless of whether we agree with how they did something. Either we follow consensus by majority of engines (and drive the spec to match consensus), or where that's not clear, we try to follow the spec as much as possible (and clarify the spec as needed).

In this case, ChakraCore's implementation being updated to match SpiderMonkey's (and as close to the spec text as we feel is possible) will make 2/4 consensus on how to format negative years with the other engines differing.

@dilijev
Copy link
Contributor

dilijev commented Nov 21, 2017

As for negative years, besides negative sign and then the absolute value padded to 4 digits, the other logical interpretation of the spec is what jsc and d8 have done:

jsc and d8 behave as printf formatting a negative number with printf("%04d", negativeNumber);, which for int negativeNumber = -3; would print -003. (Java's String.format also agrees with that interpretation of %04d with a negative argument.)

This is what d8 and jsc have done in terms of output, and I'd imagine that the implemented the formatting behavior with something akin to sprintf instead of rolling their own as we've done here.

We don't have to change this PR at this time since it's unclear from the spec, but we should bring up this point as a point that lacks clarity in the spec.

If the spec clarification ends up favoring -003 over -0003 then it will be easy to make that change later.

dilijev added a commit to dilijev/ecma262 that referenced this pull request Nov 21, 2017
…ototype.toUTCString

Note: the other instance of _month_ before _day_ (in #sec-datestring) is correct w.r.t. web reality (4/4).

See the test case below for the web reality (4/4 engines) output for all 4 `Date.prototype.to*String()` method outputs:

    ## Source
    print(new Date('2008-10-13T05:16:33Z').toUTCString());
    print(new Date('2008-10-13T05:16:33Z').toString());
    print(new Date('2008-10-13T05:16:33Z').toDateString());
    print(new Date('2008-10-13T05:16:33Z').toTimeString());

    #### ch, d8, jsc, sm
    Mon, 13 Oct 2008 05:16:33 GMT
    Sun Oct 12 2008 22:16:33 GMT-0700 (Pacific Daylight Time)
    Sun Oct 12 2008
    22:16:33 GMT-0700 (Pacific Daylight Time)

See discussion at chakra-core/ChakraCore#4067 (comment)
@xiaoyinl xiaoyinl changed the title Make Date.to(UTC)String() outputs match SpiderMonkey Make Date.to(UTC)String() outputs match web reality and spec language Nov 21, 2017
@dilijev
Copy link
Contributor

dilijev commented Nov 21, 2017

These two tests failed. They are tagged exclude_jenkins (timezone sensitive) so they didn't run on GitHub's CI. Can you look into the failures?

x64_debug\dynapogo\rl.results.log:Date (DateParse2.js () exe) -- failed : exec time=0
x64_debug\dynapogo\rl.results.log:Date (toStringAndToUTCStringYearPadding.js () exe) -- failed : exec time=0

@xiaoyinl
Copy link
Contributor Author

I have fixed the test failure for toStringAndToUTCStringYearPadding.js. The other failure is a little complicated. It's due to the change of the BC year to negative year. DateParse2.js tests if Date.parse is able to parse outputs of Date.toString() and Date.toUTCString(). The thing is the current Date.parse can parse string like Tue, 02 Feb 2013 B.C. 09:02:03 GMT, but not Tue, 02 Feb -2012 09:02:03 GMT. Thus the change in this PR breaks the test.

But as we discussed earlier fixing Date.parse isn't the scope of this PR, so I wonder if I can disable DateParse2.js for now, and I will send another PR to fix that and at that time re-enable this test?

@dilijev
Copy link
Contributor

dilijev commented Nov 21, 2017

@xiaoyinl That sounds fair. You can disable just the smallest part of that test that fails (by commenting it out). Also open an issue to track the disabled portion of the test.

1. If the absolute value of the year is less than 1000, pad it to 4 digits
2. For a BC year, output negative year, rather than positive year + "B.C."

Some tests in DateParse2.js fail due to the second change. They will be re-enabled
after #4178 is fixed.
@xiaoyinl
Copy link
Contributor Author

@dilijev I commented out 2 tests and removed corresponding outputs in the baseline file. I run the tests locally and all tests are passed now.

@dilijev
Copy link
Contributor

dilijev commented Nov 22, 2017

Looks great! I'll merge soon.

@xiaoyinl
Copy link
Contributor Author

@dilijev Thank you!

@chakrabot chakrabot merged commit 3db8983 into chakra-core:release/1.8 Nov 22, 2017
chakrabot pushed a commit that referenced this pull request Nov 22, 2017
…eality and spec language

Merge pull request #4067 from xiaoyinl:parse_own_toString

Ref #4178

[Sec 20.3.3.2](https://tc39.github.io/ecma262/#sec-date.parse) requires `Date.parse` should be able to parse the values returned by `Date.toString()`, `Date.toUTCString()` and `Date.toISOString()` in each engine's own implementation. In ChakraCore, if the year is <100, `toString()` and `toUTCString()` don't pad the year to 4 digits, and thus `Date.parse` can't parse them.

This PR makes two changes:
1. If the absolute value of the year is less than 1000, pad it to 4 digits
2. For a BC year, output negative year, rather than positive year + "B.C."
chakrabot pushed a commit that referenced this pull request Nov 22, 2017
…s match web reality and spec language

Merge pull request #4067 from xiaoyinl:parse_own_toString

Ref #4178

[Sec 20.3.3.2](https://tc39.github.io/ecma262/#sec-date.parse) requires `Date.parse` should be able to parse the values returned by `Date.toString()`, `Date.toUTCString()` and `Date.toISOString()` in each engine's own implementation. In ChakraCore, if the year is <100, `toString()` and `toUTCString()` don't pad the year to 4 digits, and thus `Date.parse` can't parse them.

This PR makes two changes:
1. If the absolute value of the year is less than 1000, pad it to 4 digits
2. For a BC year, output negative year, rather than positive year + "B.C."
@xiaoyinl xiaoyinl deleted the parse_own_toString branch November 22, 2017 03:20
dilijev added a commit to dilijev/ChakraCore that referenced this pull request Jan 12, 2018
… Date parse implementation.

See:

* [ 30082da ] 2017-11-21 17:57 doilij@m.. [1.8>master] [MERGE chakra-core#4067 @xiaoyinl] Make Date.to(UTC)String() outputs match web reality and spec language
* [ 03db098 ] 2017-11-21 14:58 doilij@m.. [1.8>master] [MERGE chakra-core#3988 @xiaoyinl] Make Date.parse() support milliseconds (Fix chakra-core#3980)
* [ afc6789 ] 2017-10-26 19:19 doilij@m.. [MERGE chakra-core#4062 @xiaoyinl] treat two digit years less than 50 as 21st century years

Related: Issue chakra-core#4543: Refactor test parseISO.js to increase test coverage
chakrabot pushed a commit that referenced this pull request Jan 13, 2018
…e for recent changes to Date parse implementation.

Merge pull request #4544 from dilijev:Bug15077412-parseISO

See:

* [ 30082da ] 2017-11-21 17:57 doilij@m.. [1.8>master] [MERGE #4067 @xiaoyinl] Make Date.to(UTC)String() outputs match web reality and spec language
* [ 03db098 ] 2017-11-21 14:58 doilij@m.. [1.8>master] [MERGE #3988 @xiaoyinl] Make Date.parse() support milliseconds (Fix #3980)
* [ afc6789 ] 2017-10-26 19:19 doilij@m.. [MERGE #4062 @xiaoyinl] treat two digit years less than 50 as 21st century years

Related: Issue #4543: Refactor test parseISO.js to increase test coverage
chakrabot pushed a commit that referenced this pull request Jan 13, 2018
…seISO.baseline for recent changes to Date parse implementation.

Merge pull request #4544 from dilijev:Bug15077412-parseISO

See:

* [ 30082da ] 2017-11-21 17:57 doilij@m.. [1.8>master] [MERGE #4067 @xiaoyinl] Make Date.to(UTC)String() outputs match web reality and spec language
* [ 03db098 ] 2017-11-21 14:58 doilij@m.. [1.8>master] [MERGE #3988 @xiaoyinl] Make Date.parse() support milliseconds (Fix #3980)
* [ afc6789 ] 2017-10-26 19:19 doilij@m.. [MERGE #4062 @xiaoyinl] treat two digit years less than 50 as 21st century years

Related: Issue #4543: Refactor test parseISO.js to increase test coverage
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants