Unresolved element found in Spring DefaultListableBeanFactory

[dsyer]

Latest efforts to make a non-trivial Spring application work with native-image are now failing at runtime with this:

Caused by: com.oracle.svm.core.jdk.UnsupportedFeatureError: Unresolved element found 
	at java.lang.Throwable.<init>(Throwable.java:265)
	at java.lang.Error.<init>(Error.java:70)
	at com.oracle.svm.core.jdk.UnsupportedFeatureError.<init>(UnsupportedFeatureError.java:31)
	at com.oracle.svm.core.jdk.Target_com_oracle_svm_core_util_VMError.unsupportedFeature(VMErrorSubstitutions.java:100)
	at com.oracle.svm.core.snippets.SnippetRuntime.unresolved(SnippetRuntime.java:205)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1056)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveDependency(AbstractAutowireCapableBeanFactory.java:346)
	at com.example.func.ObjectProviders$LazyObjectProvider.provider(ObjectProviders.java:120)
...

It could be that there is still something we can workaround or configure here, but the error isn't telling me enough to know one way or the other. To reproduce build the "jetty" branch of this project and use a graalVM build from master (to work around #497), then attempt to build a native image as in the README, but from the FuncApplication instead of MicroApplication.

[mukel]

I tried to reproduce using this, without success; after adding some tweaks to the reflection config, here's the log:
The error seems legit.

./func
Jul 03, 2018 4:01:40 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext@2ee868c3: startup date [Tue Jul 03 16:01:40 CEST 2018]; root of context hierarchy
Jul 03, 2018 4:01:40 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath
WARNING: Empty contextPath
Jul 03, 2018 4:01:40 PM org.springframework.boot.web.embedded.jetty.JettyReactiveWebServerFactory createJettyServer
INFO: Server initialized with port: 8080
Jul 03, 2018 4:01:40 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm null
Jul 03, 2018 4:01:40 PM org.eclipse.jetty.server.handler.ContextHandler doStart
INFO: Started o.e.j.s.ServletContextHandler@b7b7883{/,null,AVAILABLE}
Jul 03, 2018 4:01:40 PM org.eclipse.jetty.server.Server doStart
INFO: Started @6ms
Jul 03, 2018 4:01:40 PM org.springframework.context.support.AbstractApplicationContext refresh
WARNING: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.boot.web.reactive.error.ErrorWebExceptionHandler': Unexpected exception during bean creation; nested exception is java.lang.IllegalStateException: Cannot resolve constructor
Jul 03, 2018 4:01:40 PM org.eclipse.jetty.server.handler.ContextHandler doStop
INFO: Stopped o.e.j.s.ServletContextHandler@b7b7883{/,null,UNAVAILABLE}
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.lang.Throwable.<init>(Throwable.java:310)
        at java.lang.Exception.<init>(Exception.java:102)
        at java.lang.ReflectiveOperationException.<init>(ReflectiveOperationException.java:89)
        at java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:72)
        at com.oracle.svm.reflect.proxies.Proxy_1_FuncApplication_main.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:173)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.boot.web.reactive.error.ErrorWebExceptionHandler': Unexpected exception during bean creation; nested exception is java.lang.IllegalStateException: Cannot resolve constructor
        at java.lang.Throwable.<init>(Throwable.java:265)
        at java.lang.Exception.<init>(Exception.java:66)
        at java.lang.RuntimeException.<init>(RuntimeException.java:62)
        at org.springframework.core.NestedRuntimeException.<init>(NestedRuntimeException.java:56)
        at org.springframework.beans.BeansException.<init>(BeansException.java:40)
        at org.springframework.beans.FatalBeanException.<init>(FatalBeanException.java:35)
        at org.springframework.beans.factory.BeanCreationException.<init>(BeanCreationException.java:98)
        at org.springframework.beans.factory.BeanCreationException.<init>(BeanCreationException.java:114)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
        at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$468/1553573726.getObject(Unknown Source)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:760)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
        at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:61)
        at com.example.func.FuncApplication.run(FuncApplication.java:112)
        at com.example.func.FuncApplication.main(FuncApplication.java:93)
        ... 3 more
Caused by: java.lang.IllegalStateException: Cannot resolve constructor
        at java.lang.Throwable.<init>(Throwable.java:287)
        at java.lang.Exception.<init>(Exception.java:84)
        at java.lang.RuntimeException.<init>(RuntimeException.java:80)
        at java.lang.IllegalStateException.<init>(IllegalStateException.java:75)
        at com.example.func.ObjectProviders$LazyObjectProvider.constructor(ObjectProviders.java:150)
        at com.example.func.ObjectProviders$LazyObjectProvider$$Lambda$496/921671971.apply(Unknown Source)
        at java.util.HashMap.computeIfAbsent(HashMap.java:1127)
        at com.example.func.ObjectProviders$LazyObjectProvider.provider(ObjectProviders.java:114)
        at com.example.func.ObjectProviders$LazyObjectProvider.getIfAvailable(ObjectProviders.java:99)
        at org.springframework.beans.factory.ObjectProvider.getIfAvailable(ObjectProvider.java:69)
        at org.springframework.boot.autoconfigure.web.reactive.error.ErrorWebFluxAutoConfiguration.<init>(ErrorWebFluxAutoConfiguration.java:77)
        at com.example.func.FuncApplication.errorWebFluxAutoConfiguration(FuncApplication.java:214)
        at com.example.func.FuncApplication.lambda$registerErrorWebFluxAutoConfiguration$13(FuncApplication.java:204)
        at com.example.func.FuncApplication$$Lambda$406/470036539.get(Unknown Source)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.obtainFromSupplier(AbstractAutowireCapableBeanFactory.java:1162)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1110)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:543)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:503)
        ... 14 more
Caused by: java.lang.NoSuchMethodException: org.springframework.boot.autoconfigure.web.reactive.error.ErrorWebFluxAutoConfiguration.<init>()
        at java.lang.Throwable.<init>(Throwable.java:265)
        at java.lang.Exception.<init>(Exception.java:66)
        at java.lang.ReflectiveOperationException.<init>(ReflectiveOperationException.java:56)
        at java.lang.NoSuchMethodException.<init>(NoSuchMethodException.java:51)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getConstructor(Class.java:1825)
        at com.example.func.ObjectProviders$LazyObjectProvider.constructor(ObjectProviders.java:147)
        ... 27 more

Maybe I'm missing something from the jetty branch, can you please make sure is up to date and provide more details on how to reproduce?

[dsyer]

Sorry, you were missing some additional JSON config for the reflection access. Try again now. And build with mvn install -U to get the latest Spring snapshots.

[cstancu]

@dsyer I was able to reproduce your error. The error is caused by an unresolved element, i.e., it is most likely an error that was pushed to runtime by using --report-unsupported-elements-at-runtime. The error is thrown from org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency. If you run without --report-unsupported-elements-at-runtime and search for DefaultListableBeanFactory.resolveDependency in the list of unsupported elements you will find the missing class, DefaultListableBeanFactory$Jsr330DependencyProvider:

Error: Bytecode parsing error: java.lang.NoClassDefFoundError: org/springframework/beans/factory/support/DefaultListableBeanFactory$Jsr330DependencyProvider
Trace: 
	at parsing org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1056)
Call path from entry point to org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DependencyDescriptor, String, Set, TypeConverter): 
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1050)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveDependency(AbstractAutowireCapableBeanFactory.java:346)
	at com.example.func.ObjectProviders$LazyObjectProvider.provider(ObjectProviders.java:120)
	at com.example.func.ObjectProviders$LazyObjectProvider.getIfAvailable(ObjectProviders.java:98)
	at org.springframework.boot.autoconfigure.web.reactive.WebFluxAutoConfiguration$WebFluxConfig.<init>(WebFluxAutoConfiguration.java:128)
	at com.example.func.FuncApplication.lambda$registerWebFluxAutoConfiguration$33(FuncApplication.java:282)
	at com.example.func.FuncApplication$$Lambda$2256/1792873433.get(Unknown Source)
	at com.oracle.svm.core.jdk.SystemPropertiesSupport.initializeLazyValue(SystemPropertiesSupport.java:121)
	at com.oracle.svm.core.jdk.SystemPropertiesSupport.setProperty(SystemPropertiesSupport.java:106)
	at com.oracle.svm.core.jdk.Target_java_lang_System.setProperty(JavaLangSubstitutions.java:357)
	at com.oracle.svm.core.properties.RuntimePropertyParser.parseProperty(RuntimePropertyParser.java:68)
	at com.oracle.svm.core.properties.RuntimePropertyParser.parse(RuntimePropertyParser.java:43)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:160)
	at com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)

Unfortunatelly we cannot print the actual unresolved element currently since we rely on the Graal deoptimization mechanism when pushing the resolution errors to runtime, and Graal doesn't keep track of which elements are missing, it deoptimizes anyway. However, an Unresolved element found thrown at runtime from a native image is always (as far as I am aware) inserted by --report-unsupported-elements-at-runtime.

[sdeleuze]

Useful tip thanks, indeed DefaultListableBeanFactory$Jsr330DependencyProvider was missing, we are going to see how far we go with additional configuration.

[dsyer]

When I compile without the --report-unsupported-elements-at-runtime I do see that particular error, but I also see a load of false positives, which makes it kind of useless for this kind of app. AFAIK Jsr330DependencyProvider is a false positive too (or should be) since it is not actually used at runtime. Indeed that line in the source code does not refer to the "missing" type. So something is still wrong here, or I fail to understand.

[cstancu]

The actual missing class is javax.inject.Provider.

We call jdk.vm.ci.meta.ResolvedJavaType.getEnclosingClass() on org.springframework.beans.factory.support.DefaultListableBeanFactory$DependencyObjectProvider during classpath scanning. This calls jdk.vm.ci.hotspot.HotSpotResolvedObjectTypeImpl.getEnclosingType() which eventually calls java.lang.Class.getEnclosingClass(), which throws java.lang.NoClassDefFoundError: org/springframework/beans/factory/support/DefaultListableBeanFactory$Jsr330DependencyProvider. This seemed weird. The enclosing class is org.springframework.beans.factory.support.DefaultListableBeanFactory, and Jsr330DependencyProvider, also a subclass a DefaultListableBeanFactory, is on the classpath as well. To check if the "missing" class is actually missing and to check if there is something weird in the bytecode I run javap -c -classpath $CP org.springframework.beans.factory.support.DefaultListableBeanFactory.Jsr330DependencyProvider and the class is there.

Digging further I run this small test:

public class Test {
	public static void main(String[] args) throws Exception {
		Class<?> clazz = Class.forName("org.springframework.beans.factory.support.DefaultListableBeanFactory$DependencyObjectProvider");
		System.out.println(clazz.getEnclosingClass());
	}
}

After I compile and run it using the same $CP I get:

$ javac -cp $CP Test.java
$ java -cp $CP Test
Exception in thread "main" java.lang.NoClassDefFoundError: javax/inject/Provider
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.getDeclaringClass0(Native Method)
	at java.lang.Class.getDeclaringClass(Class.java:1235)
	at java.lang.Class.getEnclosingClass(Class.java:1277)
	at Test.main(Test.java:6)
Caused by: java.lang.ClassNotFoundException: javax.inject.Provider
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 16 more

Indeed javax.inject.Provider is not on the classpath:

$ javap -c -classpath $CP javax.inject.Provider
Error: class not found: javax.inject.Provider

Looking at the DefaultListableBeanFactory code I see that Jsr330DependencyProvider implements javax.inject.Provider. So when we ask HotSpot for the enclosing class of DefaultListableBeanFactory$DependencyObjectProvider it tries to load DefaultListableBeanFactory. While loading DefaultListableBeanFactory it also loads its subclasses, including DefaultListableBeanFactory$Jsr330DependencyProvider which fails, therefore the error.

DefaultListableBeanFactory has a fallback for when javax.inject.Provider is missing:

	@Nullable
	private static Class<?> javaxInjectProviderClass;

	static {
		try {
			javaxInjectProviderClass =
					ClassUtils.forName("javax.inject.Provider", DefaultListableBeanFactory.class.getClassLoader());
		}
		catch (ClassNotFoundException ex) {
			// JSR-330 API not available - Provider interface simply not supported then.
			javaxInjectProviderClass = null;
		}
	}

	// ...

	else if (javaxInjectProviderClass == descriptor.getDependencyType()) {
				return new Jsr330ProviderFactory().createDependencyProvider(descriptor, requestingBeanName);
	}
	// ...

However, this doesn't help us much as we actually fail during bytecode parsing, before any constant folding is run.

[dsyer]

That is weird. So it's a "feature" of Hotspot?

Class.forName("org.springframework.beans.factory.support.DefaultListableBeanFactory")

succeeds but

Class.forName("org.springframework.beans.factory.support.DefaultListableBeanFactory$DependencyObjectProvider").getEnclosingClass())

fails. Anything we can do about it?

[cstancu]

That's the behavior on HotSpot, I guess you can call it a feature. I don't think there is anything that we can do about it in SVM. We need to load the enclosing type and build the entire type hierarchy eagerly. The only solution that I can see is refactoring the code in a way that avoids this issue.

[dsyer]

Worked around in Spring (https://jira.spring.io/browse/SPR-17014). This issue can be closed if no-one else has anything to say.