Add Client Metadata Update Support. #1708

vbabanin · 2025-05-06T07:06:54Z

Description

This PR updates the Java sync and reactive-streams driver to support dynamic client metadata updates, enabling wrapping libraries and integrations to append metadata about themselves when they do not instantiate the MongoClient directly.

JAVA-5870

NOTE BEFORE MERGING:

Remove a submodule link to a fork: #1708 (comment)

JAVA-5854

vbabanin · 2025-05-08T05:25:19Z

buildSrc/src/main/kotlin/conventions/testing-base.gradle.kts

+dependencies {
+    testImplementation(libs.assertj)
+}


AssertJ has been used in this PR and added to test-base as it is a useful library that could be shared across all modules.

A decision to use a new assertion library seems like something that should be debated a bit more widely, rather than being brought in as part of a PR like this one. How important do you think it is to the assertions required for this PR?

I agree that adopting a new assertion or testing approach should generally be a broader team decision. This isn’t a critical for this PR, so I can remove it from this PR. AssertJ has already been used in the Kotlin coroutine module (reference) and is present in the toml file, though it’s not widely used in the codebase. This PR made the library visible to all modules, based on its usefulness in the Hibernate repository, which was the motivation for extending its usage further.

For now, I will revert this change and we can revisit the discussion as a team.

JAVA-5854

JAVA-5870

jyemin

Initial feedback

driver-sync/src/main/com/mongodb/client/MongoClient.java

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

JAVA-5870

# Conflicts: # driver-core/src/test/resources/specifications # driver-core/src/test/unit/com/mongodb/internal/connection/MultiServerClusterSpecification.groovy

# Conflicts: # driver-core/src/main/com/mongodb/internal/connection/BaseCluster.java

…fication.

nhachicha

Not sure why the changes of SDAM/Connection/Monitoring is part of this PR? The client metadata aspect LGTM but didn't review the SDAM changes

nhachicha · 2025-06-18T09:32:13Z

driver-core/src/main/com/mongodb/internal/connection/AbstractMultiServerCluster.java

@@ -269,7 +270,7 @@ private boolean handleReplicaSetMemberChanged(final ServerDescription newDescrip
        }

        if (isStalePrimary(newDescription)) {
-            invalidatePotentialPrimary(newDescription);
+            invalidatePotentialPrimary(newDescription, new MongoStalePrimaryException("Primary marked stale due to electionId/setVersion mismatch"));


Is there a test asserting this exception?

Replied in #1708 (comment).

nhachicha · 2025-06-18T13:41:09Z

...-core/src/test/unit/com/mongodb/internal/connection/DefaultServerMonitorSpecification.groovy

why it show up empty in the diff?

Replied in #1708 (comment).

jyemin · 2025-06-18T14:35:25Z

buildSrc/src/main/kotlin/conventions/testing-base.gradle.kts

+dependencies {
+    testImplementation(libs.assertj)
+}


A decision to use a new assertion library seems like something that should be debated a bit more widely, rather than being brought in as part of a PR like this one. How important do you think it is to the assertions required for this PR?

jyemin · 2025-06-18T14:40:42Z

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+import static com.mongodb.internal.Locks.withLock;
+import static com.mongodb.internal.connection.ClientMetadataHelper.createClientMetadataDocument;


Now that we have a proper ClientMetadata class that is meant to be instantiated, it seems like we no longer need a separate ClientMetadataHelper class, instead moving the methods of that class into this one.

jyemin · 2025-06-18T14:56:44Z

driver-core/src/main/com/mongodb/internal/connection/ClientMetadataHelper.java

+     * <p>
+     * Platform is appended separately to the existing platform if it does not exceed {@value MAXIMUM_CLIENT_METADATA_ENCODED_SIZE} bytes.
+     */
+    public static BsonDocument updateClientMetadataDocument(final BsonDocument clientMetadataDocument,


Let's consider a slightly different algorithm for updating client metadata that shared more logic with createClientMetadataDocument.

Now that we have a ClientMetadata class that is meant to be instantiated, we can store more than just the resulting BsonDocument representing the metadata. We can also store the data from which it was created: the application name and the MongoDriverInformation. If we have that, we can share all the create logic by first merging the original MongoDriverInformation with the appended one, and then calling createClientMetadataDocument. We might need a new MongoDriverInformation#merge method, or it could be added to the MongoDriverInformation.Builder, but otherwise it reduces the code complexity a bit.

What do you think?

After our offline discussion, I checked the spec and see that we’re actually allowed to omit any values outside of client.driver if the appended driver name, version, and platform take up the full space.

I agree that removing updateClientMetadata makes code easier to reason about.

I’ve updated the code to reuse createClientMetadataDocument, but I think we should avoid adding public API elements purely for internal purposes. To address that, I added an internal class to encapsulate the metadata construction logic - similar to what we did with SearchIndexRequest as an internal representation of SearchIndexModel.

I pushed a commit to a separate branch vbabanin@a41ad61. What do you think?

I like the approach.

I added the changes to this PR in this commit: 66a5913. Thank you! Let me know if you have any further suggestions.

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

vbabanin · 2025-06-19T07:04:58Z

Not sure why the changes of SDAM/Connection/Monitoring is part of this PR? The client metadata aspect LGTM but didn't review the SDAM changes

After merging main into this branch, some diffs (SDAM-related changes) are appearing, even though they’re already present on main and the total number of line additions hasn’t changed. I’m looking into why this is happening.

nhachicha · 2025-06-23T12:31:09Z

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

+            @Override
+            boolean isCurrentRuntimeContainer() {
+                try {
+                    return Files.exists(get(File.separator + ".dockerenv"));


The file is not always guaranteed to be created. we can add other heuristics based on cgroup and environment variables (set by Kubernetes for example)

boolean isRunningInContainer() { // Check for /.dockerenv if (Files.exists(Paths.get("/.dockerenv"))) return true; // Check cgroup for Docker/container patterns try { List<String> lines = Files.readAllLines(Paths.get("/proc/1/cgroup")); for (String line : lines) { if (line.contains("docker") || line.contains("kubepods") || line.contains("containerd")) { return true; } } } catch (IOException ignored) {} // Check for common container environment variable if (System.getenv("container") != null) return true; return false; }

Those are valid points and I agree. However, these changes were just moved from ClientMetadataHelper to consolidate them in the ClientMetadata instance; they were already present in the codebase and were not introduced in this PR.

You can see the diff here:
https://github.com/mongodb/mongo-java-driver/pull/1708/files#diff-ad857628989f20f23a2f410c3a2c9b49d1fc664d06a2998a489393adb5bd8e8dR161

The methods getOperatingSystemType, getOperatingSystemName, and nameStartsWith were also moved from ClientMetadataHelper. They show up as new additions because I relocated them to the bottom of the class for better organization - separating utility methods from the core logic.

nhachicha · 2025-06-23T13:09:54Z

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

+                }
+            }
+        },
+        UNKNOWN(null);


Are we interested to track other containers like Podman?

Replied in #1708 (comment).

nhachicha · 2025-06-23T13:22:54Z

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

+        K8S("kubernetes") {
+            @Override
+            boolean isCurrentOrchestrator() {
+                return FaasEnvironment.getEnv("KUBERNETES_SERVICE_HOST") != null;


Maybe also check for /etc/kubernetes per node if available?

Replied in #1708 (comment).

nhachicha · 2025-06-23T13:33:16Z

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

+                .build();
+    }
+
+    private static String listToString(final List<String> listOfStrings) {


Can't we use

String result = listOfStrings.stream() .collect(Collectors.joining(SEPARATOR));

Instead of this method?

Replied in #1708 (comment).

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java

...r-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/MongoClientImpl.java

JAVA-5870

jyemin

LGTM!

alexbevi · 2025-06-27T13:36:13Z

Adding an additional (unnecessary) LGTM to show excitement for this feature 😃

nhachicha

Nice 🎉 minor comments

nhachicha · 2025-06-30T15:22:42Z

driver-sync/src/test/functional/com/mongodb/client/AbstractClientMetadataProseTest.java

+                        builder.maxConnectionIdleTime(1, TimeUnit.MILLISECONDS))
+                .build())) {
+
+            //TODO change get() to orElseThrow


Still relevant?

nhachicha · 2025-06-30T15:33:02Z

driver-sync/src/test/functional/com/mongodb/client/AbstractClientMetadataProseTest.java

+            updateClientMetadata(driverVersion, driverName, driverPlatform, mongoClient);
+
+            //then
+            //TODO change get() to orElseThrow


nhachicha · 2025-06-30T15:36:08Z

driver-sync/src/test/functional/com/mongodb/client/AbstractClientMetadataProseTest.java

+                        builder.maxConnectionIdleTime(1, TimeUnit.MILLISECONDS))
+                .build())) {
+
+            //TODO change get() to orElseThrow


nhachicha · 2025-06-30T15:36:15Z

driver-sync/src/test/functional/com/mongodb/client/AbstractClientMetadataProseTest.java

+            updateClientMetadata(driverVersion, driverName, driverPlatform, mongoClient);
+
+            //then
+            //TODO change get() to orElseThrow


nhachicha · 2025-06-30T15:36:27Z

driver-sync/src/test/functional/com/mongodb/client/AbstractClientMetadataProseTest.java

+                .build())) {
+
+            //when
+            //TODO change get() to orElseThrow


nhachicha · 2025-06-30T15:43:56Z

driver-sync/src/test/functional/com/mongodb/client/unified/UnifiedTest.java

@@ -723,6 +723,8 @@ private OperationResult executeOperation(final UnifiedTestContext context, final
                    return clientEncryptionHelper.executeEncrypt(operation);
                case "decrypt":
                    return clientEncryptionHelper.executeDecrypt(operation);
+                case "appendMetadata":
+                    return crudHelper.executeUpdateClientMetadata(operation);


Currently, other drivers can't update their submodule deps on the sepc until they update their runner? or are they excluding the new client metadata tests?

Other drivers can update their submodule dependencies, however they will have to skip/ignore the new client metadata tests in their test runner until after they implemented the relevant changes from the epic.

# Conflicts: # driver-core/src/test/resources/specifications

vbabanin added 3 commits May 6, 2025 00:03

Add Client Metadata update support.

9753f28

Add prose tests.

889d5c8

JAVA-5854

Add prose tests.

4b3065c

JAVA-5854

vbabanin self-assigned this May 8, 2025

vbabanin commented May 8, 2025

View reviewed changes

Add ClientMetadata entity.

2a684bf

JAVA-5854

vbabanin requested review from a team and nhachicha and removed request for a team May 8, 2025 05:55

vbabanin added 7 commits May 7, 2025 23:54

Update tests.

28c1844

JAVA-5854

Merge branch 'main' into JAVA-5854

371ce0b

Fix tests.

972fe9d

JAVA-5854

Fix tests.

bcf9cc8

JAVA-5854

Spotless fix.

179b262

JAVA-5854

Skip tests when auth is enabled.

bc43ba0

JAVA-5870

Apply Scala spotless.

3f5fc91

JAVA-5870

vbabanin mentioned this pull request May 9, 2025

DRIVERS-2985: Allow on-demand client metadata updates after MongoClient initialization. mongodb/specifications#1798

Merged

3 tasks

vbabanin added 7 commits May 9, 2025 17:12

Update tests.

61192d8

JAVA-5870

Fix tests.

e9f8dd6

JAVA-5870

Add parametrized tests.

95c1bb1

JAVA-5870

Apply Kotlin spotless.

dd63a49

JAVA-5870

Move update to separate variables.

89d67bb

JAVA-5870

Merge branch 'main' into JAVA-5854

8a18d39

Merge branch 'main' into JAVA-5854

f296d0c

vbabanin requested a review from jyemin May 20, 2025 20:58

jyemin reviewed May 20, 2025

View reviewed changes

driver-sync/src/main/com/mongodb/client/MongoClient.java Outdated Show resolved Hide resolved

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java Outdated Show resolved Hide resolved

vbabanin added 2 commits May 20, 2025 20:04

Add lock for updates.

a8dc4fb

JAVA-5870

Clone document before passing it as an argument.

8ade58b

JAVA-5870

vbabanin requested a review from jyemin May 21, 2025 19:19

Rename to "appendMetadata".

71350fa

JAVA-5870

vbabanin added 4 commits June 16, 2025 19:47

Revert specifications submodule URL to upstream repository.

31ef18e

JAVA-5870

Update specifications submodule commit hash.

38ba5e6

Merge branch 'main' into JAVA-5854

edafc54

# Conflicts: # driver-core/src/test/resources/specifications # driver-core/src/test/unit/com/mongodb/internal/connection/MultiServerClusterSpecification.groovy

Merge branch 'main' into JAVA-5854

6aa0a1e

# Conflicts: # driver-core/src/main/com/mongodb/internal/connection/BaseCluster.java

vbabanin marked this pull request as ready for review June 17, 2025 03:02

vbabanin requested a review from a team as a code owner June 17, 2025 03:02

Remove duplicate import of CLIENT_METADATA in MultiServerClusterSpeci…

dab5451

…fication.

nhachicha reviewed Jun 18, 2025

View reviewed changes

jyemin requested changes Jun 18, 2025

View reviewed changes

Merge branch 'main' into JAVA-5854

1550122

vbabanin added 2 commits June 19, 2025 21:06

Remove ClientMetadataHelper.

b5c4c20

Update native-image.properties.

b6b4d67

nhachicha reviewed Jun 23, 2025

View reviewed changes

vbabanin added 5 commits June 24, 2025 00:10

Refactor tests to use JUnit assertions and remove unused dependencies.

d43972d

Fix test.

73bdbd8

Add internal driver information.

66a5913

Fix tests.

26246f0

Merge branch 'main' into JAVA-5854

843d890

vbabanin requested review from jyemin and nhachicha June 25, 2025 20:51

jyemin reviewed Jun 25, 2025

View reviewed changes

driver-core/src/main/com/mongodb/internal/connection/ClientMetadata.java Show resolved Hide resolved

...r-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/MongoClientImpl.java Show resolved Hide resolved

Add metadata logging.

850ebe7

JAVA-5870

vbabanin requested a review from jyemin June 26, 2025 20:38

jyemin approved these changes Jun 26, 2025

View reviewed changes

nhachicha approved these changes Jun 30, 2025

View reviewed changes

vbabanin added 3 commits June 30, 2025 12:00

Merge branch 'main' into JAVA-5854

d0cabe4

# Conflicts: # driver-core/src/test/resources/specifications

Add test assertion.

cb40b6a

Fix GSSAPI test.

c44200f

Add Client Metadata Update Support. #1708

Are you sure you want to change the base?

Add Client Metadata Update Support. #1708

Conversation

vbabanin commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

NOTE BEFORE MERGING:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vbabanin Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jyemin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nhachicha left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vbabanin Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vbabanin Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vbabanin commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vbabanin Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jyemin left a comment

Choose a reason for hiding this comment

Uh oh!

alexbevi commented Jun 27, 2025

Uh oh!

nhachicha left a comment

Choose a reason for hiding this comment

vbabanin commented May 6, 2025 •

edited

Loading

vbabanin Jun 20, 2025 •

edited

Loading

vbabanin Jun 25, 2025 •

edited

Loading

vbabanin Jun 25, 2025 •

edited

Loading

vbabanin commented Jun 19, 2025 •

edited

Loading

vbabanin Jun 24, 2025 •

edited

Loading

vbabanin Jun 30, 2025 •

edited

Loading