Xerces detection regressed in 2.68.0

[NotMyFault]

Hey @rbri,

Great to see that the Xerces detection has already been fixed in c1028b0.

Would you mind cutting a release, please? This regression currently impacts various test suites in the Jenkins project.

Thanks in advance!

[rbri]

@NotMyFault - yes is was my :-)

Have made a SNAPSHOT build (2.69.0-SNAPSHOT).
Can you please confirm the problem is gone with the snapshot.

Thanks

[rbri]

@NotMyFault
ok, will do a release tomorrow

(proud to help to test jenkins)

[rbri]

2.69.0 is out

[basil]

Unfortunately with 2.69.0 the Xerces detection is working on Java 11 but failing on Java 17 as can be observed running hudson.model.ApiTest#custom_notExposedToIFrame:

[ERROR] hudson.model.ApiTest.custom_notExposedToIFrame  Time elapsed: 3.623 s  <<< ERROR!
java.lang.IllegalAccessError: class com.gargoylesoftware.htmlunit.platform.util.XmlUtilsSunXercesHelper (in unnamed module @0x1a677343) cannot access class com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl (in module java.xml) because module java.xml does not export com.sun.org.apache.xerces.internal.dom to unnamed module @0x1a677343
        at com.gargoylesoftware.htmlunit.platform.util.XmlUtilsSunXercesHelper.getAttributesOrderMap(XmlUtilsSunXercesHelper.java:46)
        at com.gargoylesoftware.htmlunit.platform.Platform.getAttributesOrderMap(Platform.java:102)
        at com.gargoylesoftware.htmlunit.util.XmlUtils.getAttributesOrderMap(XmlUtils.java:410)
        at com.gargoylesoftware.htmlunit.xml.XmlPage.<init>(XmlPage.java:150)
        at com.gargoylesoftware.htmlunit.xml.XmlPage.<init>(XmlPage.java:102)
        at com.gargoylesoftware.htmlunit.xml.XmlPage.<init>(XmlPage.java:71)
        at com.gargoylesoftware.htmlunit.DefaultPageCreator.createXmlPage(DefaultPageCreator.java:354)
        at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:225)
        at org.jvnet.hudson.test.HudsonPageCreator.createPage(HudsonPageCreator.java:54)
        at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:690)
        at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:592)
        at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:510)
        at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:417)
        at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:555)
        at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:537)
        at org.jvnet.hudson.test.JenkinsRule$WebClient.goTo(JenkinsRule.java:2610)
        at hudson.model.ApiTest.ensureXmlIsNotExposedToIFrame(ApiTest.java:268)
        at hudson.model.ApiTest.custom_notExposedToIFrame(ApiTest.java:196)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:608)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.lang.Thread.run(Thread.java:833)

The following patch seems to solve the issue

diff --git a/src/main/java/com/gargoylesoftware/htmlunit/platform/Platform.java b/src/main/java/com/gargoylesoftware/htmlunit/platform/Platform.java
index aa0814e7ad..07b191dee6 100644
--- a/src/main/java/com/gargoylesoftware/htmlunit/platform/Platform.java
+++ b/src/main/java/com/gargoylesoftware/htmlunit/platform/Platform.java
@@ -51,7 +51,7 @@ public final class Platform {
             HelperSunXerces_ = (XmlUtilsHelperAPI)
                     Class.forName("com.gargoylesoftware.htmlunit.platform.util.XmlUtilsSunXercesHelper").newInstance();
         }
-        catch (final Exception e) {
+        catch (final Exception | LinkageError e) {
             // ignore
         }
 
@@ -60,7 +60,7 @@ public final class Platform {
             HelperXerces_ = (XmlUtilsHelperAPI)
                     Class.forName("com.gargoylesoftware.htmlunit.platform.util.XmlUtilsXercesHelper").newInstance();
         }
-        catch (final Exception e2) {
+        catch (final Exception | LinkageError e2) {
             // ignore
         }
     }
diff --git a/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsSunXercesHelper.java b/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsSunXercesHelper.java
index 84a2b2dee5..2001c29660 100644
--- a/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsSunXercesHelper.java
+++ b/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsSunXercesHelper.java
@@ -18,6 +18,7 @@ import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Objects;
 
 import org.w3c.dom.Document;
 import org.w3c.dom.NamedNodeMap;
@@ -38,6 +39,11 @@ public final class XmlUtilsSunXercesHelper implements XmlUtilsHelperAPI {
 
     // private static final Log LOG = LogFactory.getLog(XmlUtilsXerces.class);
 
+    public XmlUtilsSunXercesHelper() {
+        // Force eager loading of classes in order to flush out any linkage errors early
+        Objects.hash(DeferredDocumentImpl.class, DeferredNode.class);
+    }
+
     /**
      * {@inheritDoc}
      */
diff --git a/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsXercesHelper.java b/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsXercesHelper.java
index 1c2d7f0746..787b2a99c9 100644
--- a/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsXercesHelper.java
+++ b/src/main/java/com/gargoylesoftware/htmlunit/platform/util/XmlUtilsXercesHelper.java
@@ -18,6 +18,7 @@ import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Objects;
 
 import org.apache.xerces.dom.DeferredDocumentImpl;
 import org.apache.xerces.dom.DeferredNode;
@@ -38,6 +39,11 @@ public final class XmlUtilsXercesHelper implements XmlUtilsHelperAPI {
 
     // private static final Log LOG = LogFactory.getLog(XmlUtilsXerces.class);
 
+    public XmlUtilsXercesHelper() {
+        // Force eager loading of classes in order to flush out any linkage errors early
+        Objects.hash(DeferredDocumentImpl.class, DeferredNode.class);
+    }
+
     /**
      * {@inheritDoc}
      */

Once you have a solution committed, let me know and I will try it against hudson.model.ApiTest#custom_notExposedToIFrame on Java 11 and 17. In the meantime we are downgrading back to 2.67.0 (for the second time).

[rbri]

@basil sorry for that.
Have used your suggestion to fix this. Will make a new snapshot soon. And a new release is planned for this weekend.
Any chance to test this today?

[rbri]

@basil @NotMyFault
just made a new snapshot 2.70.0-SNAPSHOT including the fix you suggested.

[basil]

Any chance to test this today?

Thank you, I tested the new version by running hudson.model.ApiTest#custom_notExposedToIFrame successfully on both Java 11 and 17 (in each case both with and without Xerces on the classpath).

It would be nice if you could delete the calls to Class.forName("com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl"); and Class.forName("org.apache.xerces.dom.DeferredDocumentImpl");, as they are now made redundant by my new logic to call Objects.hash, and the second of them also prevent consumers from shading Xerces in a relocated package namespace. We build a shaded version of HtmlUnit with third-party libraries relocated, so effectively this means that we won't be able to benefit from the Xerces support in XmlUtilsXercesHelper.

[rbri]

Done - snapshot updated.

[rbri]

@basil thanks for the hint!

[basil]

You are welcome. I just repeated the same matrix of 4 tests, and all works well. If you are ready for a release, go for it! 👍