Unable to deploy simple ear from command line on GlassFish 6 build

[starksm64]

When attempting to deploy a simple Jakarta EE 9 based ear using asadmin, a 500 error is seen due to an internal org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: javax/xml/datatype/DatatypeConfigurationException error.

Environment Details

• GlassFish Version (and build number):
  GlassFish Version: Eclipse GlassFish 6.0.0 (build starksm-private)
• JDK version:
  java version "1.8.0_251"
  Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
  Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)
• OS:
  OSX 10.14.6
• Database:
  none

---

Problem Description

Attempting to deploy a simple ear containing a stateless EJB and simple Servlet from the command line fails with an internal error.

The server shows:

Caused by: java.lang.NoClassDefFoundError: javax/xml/datatype/DatatypeConfigurationException
        at jakarta.xml.bind.DatatypeConverter.initConverter(DatatypeConverter.java:125)
        at jakarta.xml.bind.DatatypeConverter.printBase64Binary(DatatypeConverter.java:596)
        at com.sun.enterprise.admin.util.CachedCommandModel.computeETag(CachedCommandModel.java:144)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.validateCommandModelETag(CommandRunnerImpl.java:188)
        at org.glassfish.admin.rest.resources.admin.CommandResource.checkCommandModelETag(CommandResource.java:316)
        at org.glassfish.admin.rest.resources.admin.CommandResource.executeCommand(CommandResource.java:368)
        at org.glassfish.admin.rest.resources.admin.CommandResource.execCommandSimpInMultOut(CommandResource.java:211)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

The full stack trace is attached:
full-strace.txt

Steps to reproduce

Using the attached test.ear.zip, copy it into /tmp/test.ear, and use the asadmin command from the glassfish6 build:

glassfish$ bin/asadmin deploy /tmp/test.ear
I/O Error: Server returned HTTP response code: 500 for URL: http://localhost:4848/command/deploy
Command deploy failed.

Impact of Issue

Unable to validate bean-validation TCK due to being unable to deploy test EARs.

[jGauravGupta]

Hi @scottkurz,

I tried to reproduce the issue on GlassFish master and got a different issue on the deployment of the test.ear file:

[2020-06-08T11:16:04.080+0530] [glassfish 6.0] [SEVERE] [NCLS-CORE-00029] [javax.enterprise.system.core] [tid: _ThreadID=54 _ThreadName=admin-listener(1)] [timeMillis: 1591595164080] [levelValue: 1000] [[
  Cannot start container ejb, exception: A MultiException has 7 exceptions.  They are:
1. com.sun.enterprise.module.ResolveError: Failed to start OSGiModuleImpl:: Bundle = [org.glassfish.main.orb.iiop [218]], State = [NEW]
2. java.lang.IllegalStateException: Could not load descriptor SystemDescriptor(
	implementation=org.glassfish.enterprise.iiop.impl.GlassFishORBFactoryImpl
	contracts={org.glassfish.enterprise.iiop.impl.GlassFishORBFactoryImpl,org.glassfish.enterprise.iiop.api.GlassFishORBFactory}
	scope=jakarta.inject.Singleton
	qualifiers={}
	descriptorType=CLASS
	descriptorVisibility=NORMAL
	metadata=Bundle-SymbolicName={org.glassfish.main.orb.iiop},Bundle-Version={6.0.0.SNAPSHOT}
	rank=0
	loader=OsgiPopulatorPostProcessor.HK2Loader(OSGiModuleImpl:: Bundle = [org.glassfish.main.orb.iiop [218]], State = [NEW],697511667)
	proxiable=null
	proxyForSameScope=null
	analysisName=null
	id=110
	locatorId=0
	identityHashCode=1579516581
	reified=false)
3. java.lang.IllegalStateException: Unable to perform operation: post construct on org.glassfish.enterprise.iiop.api.GlassFishORBHelper
4. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of com.sun.ejb.containers.EjbContainerUtilImpl errors were found
5. java.lang.IllegalStateException: Unable to perform operation: resolve on com.sun.ejb.containers.EjbContainerUtilImpl
6. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of org.glassfish.ejb.startup.EjbContainerStarter errors were found
7. java.lang.IllegalStateException: Unable to perform operation: resolve on org.glassfish.ejb.startup.EjbContainerStarter
]]
Aborting, Failed to start container org.glassfish.ejb.startup.EjbContainerStarter
  Error occurred
java.lang.Exception: Aborting, Failed to start container org.glassfish.ejb.startup.EjbContainerStarter
	at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:715)
	at org.glassfish.javaee.full.deployment.EarDeployer.prepareBundle(EarDeployer.java:275)
	at org.glassfish.javaee.full.deployment.EarDeployer.access$200(EarDeployer.java:64)
	at org.glassfish.javaee.full.deployment.EarDeployer$1.doBundle(EarDeployer.java:131)
        ....

After applying the ORB patch (eclipse-ee4j/orb#97), The ear deploys successfully and GreeterServlet is accessible via http://localhost:8080/test/Greeter:

[smillidge]

The Orb issue should be fixed in the latest master.

[starksm64]

I was still seeing the problem after pulling down the latest updates and doing a clean build. I then threw away all of the local maven repo jakarta artifacts and rebuilt, and now the deployment is working. Either some cached staging build or other local build was causing the problem.