Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

30 standalone tags (jstl) failures #631

Closed
arjantijms opened this issue Mar 1, 2021 · 23 comments
Closed

30 standalone tags (jstl) failures #631

arjantijms opened this issue Mar 1, 2021 · 23 comments
Assignees
Labels
9.1 9.1 release

Comments

@arjantijms
Copy link
Contributor

Describe the bug
30 standalone Tags (JSTL) TCK tests are failing when run in JDK11.

Errors like:

HTTP Status 500 - Internal Server Error</h1><hr/><p><b>type</b> Exception report</p><p><b>message</b>Internal Server Error</p><p><b>description</b>The server encountered an internal error that prevented it from fulfilling this request.</p><p><b>exception</b> <pre>jakarta.servlet.ServletException: In &amp;lt&#59;parseDate&amp;gt&#59;, value attribute can not be parsed: &quot;Nov 21, 2000 3:45:02 AM&quot;</pre></p><p><b>root cause</b> <pre>java.text.ParseException: Unparseable date: &quot;Nov 21, 2000 3:45:02 AM&quot;</pre></p><p><b>note</b> <u>The full stack traces of the exception and its root causes are available in the Eclipse GlassFish  6.1.0  logs.</u></p><hr/><h3>Eclipse GlassFish  6.1.0 </h3></body></html>
[javatest.batch] 03-01-2021 17:03:04:  ERROR: Test case throws exception: [BaseUrlClient] null failed! Check output for cause of failure.
[javatest.batch] 03-01-2021 17:03:04:  ERROR: Exception at:  
[javatest.batch] 03-01-2021 17:03:04:  ERROR: com.sun.ts.tests.common.webclient.TestFailureException: Test FAILED!
[javatest.batch] 	at com.sun.ts.tests.common.webclient.WebTestCase.execute(WebTestCase.java:178)
[javatest.batch] 	at com.sun.ts.tests.common.webclient.BaseUrlClient.invoke(BaseUrlClient.java:456)
[javatest.batch] 	at com.sun.ts.tests.jstl.compat.onedotzero.JSTLClient.positiveFormatLocalizationContextI18NTest(JSTLClient.java:200)
[javatest.batch] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[javatest.batch] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[javatest.batch] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[javatest.batch] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:596)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:337)
[javatest.batch] 	at com.sun.ts.tests.jstl.compat.onedotzero.JSTLClient.run(JSTLClient.java:65)
[javatest.batch] 	at com.sun.ts.tests.jstl.compat.onedotzero.JSTLClient.main(JSTLClient.java:51)

To Reproduce
Run https://ci.eclipse.org/glassfish/view/TCK/job/run_tags_tck_against_staged_build/

Expected behavior
100% pass, 0% errors

@arjantijms arjantijms added the 9.1 9.1 release label Mar 1, 2021
@scottmarlow
Copy link
Contributor

I'll try looking for a server.log in https://ci.eclipse.org/jakartaee-tck/job/9.1/job/eftl-standalonetck-build-run-910/8/artifact/tags-tck-results.tar.gz to see if the java.text.ParseException: Unparseable date occurs there.

@scottmarlow
Copy link
Contributor

scottmarlow commented Mar 1, 2021

From server.log I see:

[2021-02-28T18:56:06.635+0000] [glassfish 6.1] [WARNING] [] [jakarta.enterprise.web] [tid: _ThreadID=32 _ThreadName=http-listener-1(2)] [timeMillis: 1614538566635] [levelValue: 900] [[
  StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exception
java.text.ParseException: Unparseable date: "Nov 21, 2000 3:45:02 AM"
        at java.base/java.text.DateFormat.parse(DateFormat.java:395)
        at org.apache.taglibs.standard.tag.common.fmt.ParseDateSupport.doEndTag(ParseDateSupport.java:178)
        at org.apache.jsp.positiveFormatLocalizationContextI18NTest_jsp._jspx_meth_fmt_parseDate_0(positiveFormatLocalizationContextI18NTest_jsp.java:299)
        at org.apache.jsp.positiveFormatLocalizationContextI18NTest_jsp._jspService(positiveFormatLocalizationContextI18NTest_jsp.java:134)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:68)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587)
        at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:324)
        at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:355)
        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:284)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587)
        at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1540)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
        at com.sun.ts.tests.jstl.common.filters.JstlDbFilter.doFilter(JstlDbFilter.java:121)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:215)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:119)
        at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:611)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:550)
        at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:75)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:114)
        at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:332)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:199)
        at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:440)
        at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:144)
        at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:174)
        at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:153)
        at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:196)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:178)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:118)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:96)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
        at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
        at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:34)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:101)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
        at java.base/java.lang.Thread.run(Thread.java:834)

@smillidge
Copy link

Is that a test case issue then?

@alwin-joseph
Copy link
Contributor

Brian had previously mentioned in ml discussion (sub "Jakarta EE 9.1 testing with Java SE 11 - JSTL Bucket" ) that

Java 9+ changed to using CLDR for locales so there's a handful of minor differences like the above reflected throughout the results. There is a JVM option (-Djava.locale.providers=COMPAT) that reverts the behavior to what was used in Java 8, and the tests pass fine with that in place. Is that the right solution here, or should we be looking at adding/updating golden files in the TCK so they are correct for Java 11?

If adding -Djava.locale.providers=COMPAT is fine then shall we try that.

@scottmarlow
Copy link
Contributor

Brian had previously mentioned in ml discussion (sub "Jakarta EE 9.1 testing with Java SE 11 - JSTL Bucket" ) that

Java 9+ changed to using CLDR for locales so there's a handful of minor differences like the above reflected throughout the results. There is a JVM option (-Djava.locale.providers=COMPAT) that reverts the behavior to what was used in Java 8, and the tests pass fine with that in place. Is that the right solution here, or should we be looking at adding/updating golden files in the TCK so they are correct for Java 11?

If adding -Djava.locale.providers=COMPAT is fine then shall we try that.

@alwin-joseph Good point, I think that we should add -Djava.locale.providers=COMPAT for JDK11 to run in JDK 8 compatibility mode.

@scottmarlow
Copy link
Contributor

Note that the workaround was mentioned on https://www.eclipse.org/lists/jakartaee-tck-dev/msg01139.html

@pnicolucci do you agree that we should use the -Djava.locale.providers=COMPAT workaround so that JSTL tests running on JDK 11 will see JDK 8 behavior?

Please reference COMPAT mentioned on https://www.oracle.com/java/technologies/javase/jdk11-suported-locales.html. Also reference response from Lance Andersen regarding using COMPAT option until we switch to a later Java SE version for our Jakarta EE base SE version.

@arjantijms
Copy link
Contributor Author

To me it's fine, but would like to hear from @pnicolucci of course.

@brideck
Copy link
Contributor

brideck commented Mar 8, 2021

Since I originally mentioned it, I agree that the COMPAT workaround is a good path of least resistance, but I'm also going to be taking a look over the next day or so at how hard it would be to handle two sets of goldenfiles for these tests -- one for Java 8, the other for 9+.

@pnicolucci
Copy link
Contributor

I don't have a problem setting the system property if the path of least resistance is what is being looked for here. I do agree with Brian that if multiple golden files can be added and used across the various executions then it would make sense to go that route rather than setting a system property to revert to the old behavior.

@edbratt
Copy link
Contributor

edbratt commented Mar 12, 2021

So long as the property setting -Djava.locale.providers=COMPAT is documented in our test reports, we are fine. Perhaps this should be globally applied to all the TCK tests -- just so we don't trip up on this elsewhere?
Also, does this resolve all 29 failures?

@brideck
Copy link
Contributor

brideck commented Mar 12, 2021

I've been looking more closely at our results and I think that only 14 of the 29 failures could be fixed with goldenfile updates.

The other 15 fail (in Open Liberty and Glassfish) with some variation of the exception reported above:

java.text.ParseException: Unparseable date: "Nov 21, 2000 3:45:02 AM"
at java.base/java.text.DateFormat.parse(DateFormat.java:395)
at org.apache.taglibs.standard.tag.common.fmt.ParseDateSupport.doEndTag(ParseDateSupport.java:178)
...

Looking around, the consensus seems to be that using the COMPAT value is the way to get this to work. Otherwise, in this particular case I think there would need to be an update to apache.taglibs to handle the new (as of Java 9) Locales and/or migrate from using java.text to java.time for its parsing.

I can confirm that using the COMPAT property does resolve all 29 failures in Open Liberty, so one would presume it would for Glassfish as well.

@edbratt
Copy link
Contributor

edbratt commented Mar 12, 2021

Yes, probably this needs to be added for the Platform TCK test jobs as well as the the stand alone TCK test runs. Once we drop JDK8, we can refresh the tests to use the new formats. Thanks for checking for us.

@gurunrao
Copy link
Contributor

gurunrao commented Mar 15, 2021

@scottmarlow - shall we consult tags dev, on usage of COMPAT workaround to get JSTL test passed with jdk11

@pnicolucci
Copy link
Contributor

I'm not quite clear on the scenario when the ParseException is experienced. Is the date that is failing provided by the TCK?

If the tags implementation/specification had to move away from the java.text in favor of java.time then it would require quite the overhaul of the specification:

https://jakarta.ee/specifications/tags/2.0/jakarta-tags-spec-2.0.pdf

If the cases that are failing are areas where the spec document specifies java.text.DateFormat or other java.text.* classes then the spec document along with the Implementation and potentially the API would need to be reworked.

@pnicolucci
Copy link
Contributor

My assumption here is that the dates failing are unchanged from the jdk8 runs where they work. If the issue at its core is that dates that worked with JDK8 now don't work in JDK 11 due to CLDR being used as the default locale provider then IMO it makes sense to use the COMPAT option. Folks in the field using JDK 9+ would also need to do something similar if they didn't want to update their date formatting to be consistent with CLDR. At least that's my understanding of the issue at this point.

@brideck
Copy link
Contributor

brideck commented Mar 15, 2021

Looking at some of the ParseException cases more closely, I think it is indeed just a matter of how the JSPs in the tests are written.

ex.

<!-- The date to be parsed can be provided as body content to the
          action. -->
<fmt:parseDate type="both" var="e2">
    Nov 21, 2000 3:45:02 AM
</fmt:parseDate>
<fmt:formatDate value="${e2}" type="both" timeZone="EST"/>

For the above to pass with the default Java 11 options, we'd instead need a JSP using a parseDate value that looks like Nov 21, 2000, 3:45:02 AM. Which means we would need two versions of a lot of these JSPs (to support 8 and 11) and therefore two versions of the applications, then the TCK would have to sort out which one to deploy. I'm not sure how feasible that feels.

@LanceAndersen
Copy link

LanceAndersen commented Mar 15, 2021 via email

@brideck
Copy link
Contributor

brideck commented Mar 15, 2021

For the post-9.1 future, here's a branch (https://github.com/brideck/jakartaee-tck/tree/UpdateJSTLJava11Locale) containing the changes needed to get these tests to pass with Java 11's default locales and Open Liberty. They should also work for GF.

@scottmarlow
Copy link
Contributor

@brideck if you have time it could be helpful to create a pr with label =future for a later release.

@brideck
Copy link
Contributor

brideck commented Mar 16, 2021

PR for future: #664

@gurunrao
Copy link
Contributor

JSTL tests are passing with COMPAT option. WIP changes can be seen here - https://github.com/gurunrao/jakartaee-tck/commit/951e93c6ece0b4410f78e85a58434a5cb4b47295
CI run:

@gurunrao
Copy link
Contributor

PR for COMPAT option - #670

@scottmarlow
Copy link
Contributor

Passing now! Only the jaxws tests failed (could be a compilation issue I am guessing).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
9.1 9.1 release
Projects
None yet
Development

No branches or pull requests

9 participants