Skip to content

Conversation

@eubnara
Copy link
Contributor

@eubnara eubnara commented Nov 19, 2025

Description of PR

When query A record which is chained by CNAME, YARN Registry DNS Server does not properly respond. Some CNAME records are missing.

For example, "repo.maven.apache.org" is chaned as follows:

repo.maven.apache.org. 21317 IN CNAME repo.apache.maven.org.
repo.apache.maven.org. 20114 IN CNAME maven.map.fastly.net.
maven.map.fastly.net. 7 IN A 199.232.192.215
maven.map.fastly.net. 7 IN A 199.232.196.215

If ask A record for "repo.maven.apache.org" using "dig" or "nslookup", YARN Registry DNS Server will give answers similar to this:
(10.1.2.3, 10.8.8.8 IP is virtual)

$ nslookup repo.maven.apache.org 10.1.2.3
Server:		10.1.2.3
Address:	10.1.2.3#53

Non-authoritative answer:
repo.maven.apache.org	canonical name = repo.apache.maven.org.
Name:	maven.map.fastly.net
Address: 151.101.196.215
** server can't find repo.apache.maven.org: NXDOMAIN

The reason why you can see "NXDOMAIN", "nslookup" will query "A" & "AAAA" records.
If there is no answer from other dns server, "answers == null" but YARN Registry DNS Server has a bug. There is no null handling.

    // Forward lookup to primary DNS servers
    Record[] answers = getRecords(name, type);
    try {
      for (Record r : answers) {
        if (!response.findRecord(r)) {
          if (r.getType() == Type.SOA) {
            response.addRecord(r, Section.AUTHORITY);
          } else {
            response.addRecord(r, Section.ANSWER);
          }
        }
        if (r.getType() == Type.CNAME) {
          Name cname = r.getName();
          if (iterations < 6) {
            remoteLookup(response, cname, type, iterations + 1);
          }
        }
      }
    } catch (NullPointerException e) {
      return Rcode.NXDOMAIN;
    } catch (Throwable e) {
      return Rcode.SERVFAIL;
    }
    return Rcode.NOERROR;

It should be like this:

nslookup repo.maven.apache.org 10.8.8.8
Server:		10.8.8.8
Address:	10.8.8.8#53

Non-authoritative answer:
repo.maven.apache.org	canonical name = repo.apache.maven.org.
repo.apache.maven.org	canonical name = maven.map.fastly.net.
Name:	maven.map.fastly.net
Address: 151.101.196.215

How was this patch tested?

  • apply this patch to YARN Registry DNS Server in our cluster & test with dig and nslookup

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

- add chained CNAME records to answer section
- distinguish between NXDOMAIN and NOERROR + empty answer
- There is no way to figure out the type of alias from dnsjava's lookup. As a workaround, query CNAME first and then DNAME.
- dnsjava use "dnsjava.lookup.max_iterations" property in order to limit follow chaining. Default 16. Therefore, I think "iterations" var. in remoteLookup() is not necessary.
@eubnara
Copy link
Contributor Author

eubnara commented Nov 19, 2025

#3935

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 15m 4s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 33m 37s trunk passed
+1 💚 compile 16m 15s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 compile 16m 15s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 checkstyle 0m 42s trunk passed
+1 💚 mvnsite 0m 50s trunk passed
+1 💚 javadoc 0m 51s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 47s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌ spotbugs 1m 10s /branch-spotbugs-hadoop-common-project_hadoop-registry-warnings.html hadoop-common-project/hadoop-registry in trunk has 19 extant spotbugs warnings.
+1 💚 shadedclient 26m 40s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 22s the patch passed
+1 💚 compile 15m 10s the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javac 15m 10s the patch passed
+1 💚 compile 16m 16s the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 javac 16m 16s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 40s /results-checkstyle-hadoop-common-project_hadoop-registry.txt hadoop-common-project/hadoop-registry: The patch generated 15 new + 1 unchanged - 0 fixed = 16 total (was 1)
+1 💚 mvnsite 0m 47s the patch passed
-1 ❌ javadoc 0m 46s /results-javadoc-javadoc-hadoop-common-project_hadoop-registry-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt hadoop-common-project_hadoop-registry-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 5 new + 136 unchanged - 0 fixed = 141 total (was 136)
-1 ❌ javadoc 0m 46s /results-javadoc-javadoc-hadoop-common-project_hadoop-registry-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt hadoop-common-project_hadoop-registry-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 5 new + 129 unchanged - 0 fixed = 134 total (was 129)
-1 ❌ spotbugs 1m 19s /new-spotbugs-hadoop-common-project_hadoop-registry.html hadoop-common-project/hadoop-registry generated 1 new + 17 unchanged - 2 fixed = 18 total (was 19)
+1 💚 shadedclient 26m 57s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 1m 30s hadoop-registry in the patch passed.
+1 💚 asflicense 1m 1s The patch does not generate ASF License warnings.
182m 39s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-registry
Unknown bug pattern DCN_NULLPOINTER_EXCEPTION in org.apache.hadoop.registry.server.dns.RegistryDNS.getRecords(Name, int) At RegistryDNS.java:org.apache.hadoop.registry.server.dns.RegistryDNS.getRecords(Name, int) At RegistryDNS.java:[line 1190]
Subsystem Report/Notes
Docker ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8091/1/artifact/out/Dockerfile
GITHUB PR #8091
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 3dd397b0fd91 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 683b944
Default Java Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8091/1/testReport/
Max. process+thread count 640 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-registry U: hadoop-common-project/hadoop-registry
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8091/1/console
versions git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants