Query and ref are lost from `jar:` URLs that are processed by fat jar loader's handler

[TristanDupontICE]

When a ref is used in a java.net.URL, org.springframework.boot.loader.jar.Handler.parseURL() lose the information.

Code to reproduce:

URL url = new URL("jar:file:/archive.jar!/file.txt#some-ref");
System.out.println(url);

Actual output:

jar:file:/archive.jar!/file.txt

Expected output:

jar:file:/archive.jar!/file.txt#some-ref

This bug occurs only when the application is started with spring-boot-loader. Some frameworks need reference to work properly, this information cannot be lost.

[wilkinsona]

This is the same issue as was reported in #11198 but never went anywhere.

@tristandupont Can you please provide a concrete example of the need for the URL fragment? A sample that uses a framework that needs it to work properly would be ideal.

[TristanDupontICE]

Here is an simple example: https://github.com/TristanDupont/sprint_boot_url
Once started just need to call http://localhost:8080/hello

The behavior is different when the application is directly run or is run by spring-boot-loader.

(~/.javacpp/cache/ should be cleared between each test)

[wilkinsona]

Thanks for the sample. Having tweaked it to run on macOS, I believe I've reproduced the problem. The app works when run with mvn spring-boot:run but fails when run with java -jar. The failure is the following:

java.lang.UnsatisfiedLinkError: /Users/awilkinson/.javacpp/cache/openblas-0.3.0-1.4.2-macosx-x86_64.jar/org/bytedeco/javacpp/macosx-x86_64/libjniopenblas_nolapack.dylib: dlopen(/Users/awilkinson/.javacpp/cache/openblas-0.3.0-1.4.2-macosx-x86_64.jar/org/bytedeco/javacpp/macosx-x86_64/libjniopenblas_nolapack.dylib, 1): Library not loaded: @rpath/libopenblas_nolapack.dylib
  Referenced from: /Users/awilkinson/.javacpp/cache/openblas-0.3.0-1.4.2-macosx-x86_64.jar/org/bytedeco/javacpp/macosx-x86_64/libjniopenblas_nolapack.dylib
  Reason: image not found
	at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_151]
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_151]
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_151]
	at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_151]
	at java.lang.System.load(System.java:1086) ~[na:1.8.0_151]
	at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1205) ~[javacpp-1.4.2.jar!/:1.4.2]
	at org.bytedeco.javacpp.Loader.load(Loader.java:983) ~[javacpp-1.4.2.jar!/:1.4.2]
	at org.bytedeco.javacpp.Loader.load(Loader.java:882) ~[javacpp-1.4.2.jar!/:1.4.2]
	at org.bytedeco.javacpp.openblas_nolapack.<clinit>(openblas_nolapack.java:10) ~[openblas-0.3.0-1.4.2.jar!/:1.4.2]
	at java.lang.Class.forName0(Native Method) ~[na:1.8.0_151]
	at java.lang.Class.forName(Class.java:348) ~[na:1.8.0_151]
	at org.bytedeco.javacpp.Loader.load(Loader.java:941) ~[javacpp-1.4.2.jar!/:1.4.2]
	at org.bytedeco.javacpp.Loader.load(Loader.java:898) ~[javacpp-1.4.2.jar!/:1.4.2]
	at org.bytedeco.javacpp.presets.openblas_nolapack.blas_set_num_threads(openblas_nolapack.java:189) ~[openblas-0.3.0-1.4.2.jar!/:0.3.0-1.4.2]
	at sample.controller.SimpleController.healthCheck(SimpleController.java:20) ~[classes!/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_151]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_151]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_151]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_151]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:866) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851) ~[spring-webmvc-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.32.jar!/:8.5.32]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]

The relationship between the failure and the loss of the #whatever portion of the URL isn't clear from the error message and exception. Can you explain how you made that connection please?

[TristanDupontICE]

I did not look how the library use the reference, but I looked where the information is lost.

When a new java.lang.URL is created, the constructor does all the parsing job, then a java.net.URLStreamHandler is called to finish the parsing. The reference part is properly parsed by the URL.

When we start a spring boot application with the spring-boot-loader, the org.springframework.boot.loader is registered as java.protocol.handler.pkgs and org.springframework.boot.loader.jar.Handler is used as URLStreamHandler. But this handler does not use all parsed information of the URL source:

private void setFile(URL context, String file) {
	setURL(context, JAR_PROTOCOL, null, -1, null, null, normalize(file), null, null);
}

The reference information is available in the context object but the method always set a null value.

I don't know if this behavior is voluntary or if it's mistake.

[wilkinsona]

Answering my own question, the connection between the failure and the URL's ref is this line in org.bytedeco.javacpp.Loader where it creates a URL with a ref:

u = new URL(u + "#" + styles2[i]);

The ref is then used subsequently when naming the resource in the cache:

if (resourceURL.getRef() != null) {
    // ... get the URL fragment to let users rename library files ...
    String newName = resourceURL.getRef();
    // ... but create a symbolic link only if the name does not change ...
    reference = newName.equals(name);
    name = newName;
}

This is certainly an unusal usage of the URL's ref, but it's supported by the JDK's handler for jar: URLs so ours should support it too.