DATACOUCH-588 - Part 2 of framework changes. Add support for projection and distinct.

Support for projection is only for properties of the top-level entity. For instance, in UserSubmission, only the properties below can be specified in the projection. Projection support does not provide means of specifying something like address.street - you can only project (or not project) the whole address property. However, the address type in your resultType could have a subset of the properties in Address.

If the corresponding submissions in the resultType contained only the userId property

public class UserSubmission extends ComparableEntity {
	private String id;
	private String username;
	private List<String> roles;
	private Address address;
	private List<Submission> submissions;

Support for Distinct - I have appropriated the MongoDB model for Distinct.  It defines a separate DistinctOperationSupport class (within ExecutableFindByQuerySupport) which supports the distinct( distinctFields ) api and execution. The DistinctOperationSupport class has only a distinctFields member, and a 'delegate' member, which is an ExecutableFindByQuerySupport object. TBH, I don't see the advantage over simply adding a distinctFields member to ExecutableFindByQuerySupport

Amend #1 - changes as discussed in Pull Request
         - clean up test entity types

Amend #2
- Eliminate DistinctOperationSupport class. In MongoDB, only distinct on a single field is supported, so the returnType from distinct was very different from the returnType of other query operations (all(), one() etc. (but so is count(), and it doesn't need it's own class)). In Couchbase, distinct on any fields in the entity is allowed - so the returned type could be the domainType or resultType. And as(resultType) still allows any resultType to be specified. This makes it unnecessary to have combinations of interfaces such as DistinctWithProjection and DistinctWithQuery.

- Clean up the interfaces in ExecutableFindByQuery. There are two types of interfaces (a) the TerminatingFindByQuery which has the one(), oneValue() first(), firstValue(), all(), count(), exists() and stream(); and (b) the option interfaces (FindByQueryWithConsistency etc), which are essentially with-er interfaces. The changes are:
1) make all the with-er interfaces base interfaces instead of chaining them together. (I don't know why there isn't simply one interface with all the with-er methods).
2) make the ExecutableFindByQuery interface extend the Terminating interface and all the with-er interfaces.

Amend #3
- Add execution support for collections

Amend #4
- Add tests for collections. This includes a new CollectionAwareIntegrationTests class which extends a new JavaIntegratationTests class which extends the existing ClusterAwareIntegrationTests.
- Fixed up several issues collections issues that were uncovered by the tests.
- Did further cleanup of OperationSupport interfaces.

Amend #5
- Revert changes to interfaces in *Operation
- Sorted interfaces in same order for consistency (because of the chaining of interfaces, fluent methods must be called in order).

Author: mikereiche

pom.xml

@@ -197,6 +197,14 @@
             <version>${kotlin}</version>
             <scope>test</scope>
         </dependency>
+
+        <dependency>
+            <groupId>org.awaitility</groupId>
+            <artifactId>awaitility</artifactId>
+            <version>4.0.3</version>
+            <scope>test</scope>
+        </dependency>
+
     </dependencies>
 
     <repositories>

src/main/java/org/springframework/data/couchbase/core/ExecutableFindByAnalyticsOperation.java

@@ -19,12 +19,12 @@
 import java.util.Optional;
 import java.util.stream.Stream;
 
-import com.couchbase.client.java.analytics.AnalyticsScanConsistency;
-import com.couchbase.client.java.query.QueryScanConsistency;
 import org.springframework.dao.IncorrectResultSizeDataAccessException;
 import org.springframework.data.couchbase.core.query.AnalyticsQuery;
 import org.springframework.lang.Nullable;
 
+import com.couchbase.client.java.analytics.AnalyticsScanConsistency;
+
 public interface ExecutableFindByAnalyticsOperation {
 
 	/**
@@ -114,17 +114,30 @@ interface FindByAnalyticsWithQuery<T> extends TerminatingFindByAnalytics<T> {
 
 	}
 
+	@Deprecated
 	interface FindByAnalyticsConsistentWith<T> extends FindByAnalyticsWithQuery<T> {
 
 		/**
 		 * Allows to override the default scan consistency.
 		 *
 		 * @param scanConsistency the custom scan consistency to use for this analytics query.
 		 */
+		@Deprecated
 		FindByAnalyticsWithQuery<T> consistentWith(AnalyticsScanConsistency scanConsistency);
 
 	}
 
-	interface ExecutableFindByAnalytics<T> extends FindByAnalyticsConsistentWith<T> {}
+	interface FindByAnalyticsWithConsistency<T> extends FindByAnalyticsConsistentWith<T> {
+
+		/**
+		 * Allows to override the default scan consistency.
+		 *
+		 * @param scanConsistency the custom scan consistency to use for this analytics query.
+		 */
+		FindByAnalyticsConsistentWith<T> withConsistency(AnalyticsScanConsistency scanConsistency);
+
+	}
+
+	interface ExecutableFindByAnalytics<T> extends FindByAnalyticsWithConsistency<T> {}
 
 }

src/main/java/org/springframework/data/couchbase/core/ExecutableFindByAnalyticsOperationSupport.java

@@ -15,11 +15,10 @@
  */
 package org.springframework.data.couchbase.core;
 
-import org.springframework.data.couchbase.core.ReactiveFindByAnalyticsOperationSupport.ReactiveFindByAnalyticsSupport;
-
 import java.util.List;
 import java.util.stream.Stream;
 
+import org.springframework.data.couchbase.core.ReactiveFindByAnalyticsOperationSupport.ReactiveFindByAnalyticsSupport;
 import org.springframework.data.couchbase.core.query.AnalyticsQuery;
 
 import com.couchbase.client.java.analytics.AnalyticsScanConsistency;
@@ -79,10 +78,16 @@ public TerminatingFindByAnalytics<T> matching(final AnalyticsQuery query) {
 		}
 
 		@Override
+		@Deprecated
 		public FindByAnalyticsWithQuery<T> consistentWith(final AnalyticsScanConsistency scanConsistency) {
 			return new ExecutableFindByAnalyticsSupport<>(template, domainType, query, scanConsistency);
 		}
 
+		@Override
+		public FindByAnalyticsWithConsistency<T> withConsistency(final AnalyticsScanConsistency scanConsistency) {
+			return new ExecutableFindByAnalyticsSupport<>(template, domainType, query, scanConsistency);
+		}
+
 		@Override
 		public Stream<T> stream() {
 			return reactiveSupport.all().toStream();

src/main/java/org/springframework/data/couchbase/core/ExecutableFindByIdOperationSupport.java

@@ -15,12 +15,11 @@
  */
 package org.springframework.data.couchbase.core;
 
-import org.springframework.data.couchbase.core.ReactiveFindByIdOperationSupport.ReactiveFindByIdSupport;
-
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.List;
 
+import org.springframework.data.couchbase.core.ReactiveFindByIdOperationSupport.ReactiveFindByIdSupport;
 import org.springframework.util.Assert;
 
 public class ExecutableFindByIdOperationSupport implements ExecutableFindByIdOperation {

src/main/java/org/springframework/data/couchbase/core/ExecutableFindByQueryOperation.java

@@ -21,6 +21,7 @@
 
 import org.springframework.dao.IncorrectResultSizeDataAccessException;
 import org.springframework.data.couchbase.core.query.Query;
+import org.springframework.data.couchbase.core.query.QueryCriteriaDefinition;
 import org.springframework.lang.Nullable;
 
 import com.couchbase.client.java.query.QueryScanConsistency;
@@ -117,30 +118,92 @@ interface FindByQueryWithQuery<T> extends TerminatingFindByQuery<T> {
 		 */
 		TerminatingFindByQuery<T> matching(Query query);
 
+		/**
+		 * Set the filter {@link QueryCriteriaDefinition criteria} to be used.
+		 *
+		 * @param criteria must not be {@literal null}.
+		 * @return new instance of {@link ExecutableFindByQuery}.
+		 * @throws IllegalArgumentException if criteria is {@literal null}.
+		 */
+		default TerminatingFindByQuery<T> matching(QueryCriteriaDefinition criteria) {
+			return matching(Query.query(criteria));
+		}
+
 	}
 
+	@Deprecated
 	interface FindByQueryConsistentWith<T> extends FindByQueryWithQuery<T> {
 
 		/**
 		 * Allows to override the default scan consistency.
 		 *
 		 * @param scanConsistency the custom scan consistency to use for this query.
 		 */
-		FindByQueryConsistentWith<T> consistentWith(QueryScanConsistency scanConsistency);
+		@Deprecated
+		FindByQueryWithQuery<T> consistentWith(QueryScanConsistency scanConsistency);
+
+	}
+
+	interface FindByQueryWithConsistency<T> extends FindByQueryConsistentWith<T> {
+
+		/**
+		 * Allows to override the default scan consistency.
+		 *
+		 * @param scanConsistency the custom scan consistency to use for this query.
+		 */
+		FindByQueryConsistentWith<T> withConsistency(QueryScanConsistency scanConsistency);
 
 	}
 
-	interface FindByQueryInCollection<T> extends FindByQueryConsistentWith<T> {
+	interface FindByQueryInCollection<T> extends FindByQueryWithConsistency<T> {
 
 		/**
 		 * Allows to override the default scan consistency.
 		 *
 		 * @param collection the collection to use for this query.
 		 */
-		FindByQueryInCollection<T> inCollection(String collection);
+		FindByQueryWithConsistency<T> inCollection(String collection);
 
 	}
 
-	interface ExecutableFindByQuery<T> extends FindByQueryInCollection<T> {}
+	/**
+	 * Result type override (Optional).
+	 */
+	interface FindByQueryWithProjection<T> extends FindByQueryInCollection<T> {
+
+		/**
+		 * Define the target type fields should be mapped to. <br />
+		 * Skip this step if you are anyway only interested in the original domain type.
+		 *
+		 * @param resturnType must not be {@literal null}.
+		 * @param <R> result type.
+		 * @return new instance of {@link FindByQueryWithProjection}.
+		 * @throws IllegalArgumentException if resturnType is {@literal null}.
+		 */
+		<R> FindByQueryInCollection<R> as(Class<R> resturnType);
+
+	}
+
+	/**
+	 * Distinct Find support.
+	 */
+	interface FindByQueryWithDistinct<T> extends FindByQueryWithProjection<T> {
+
+		/**
+		 * Finds the distinct values for a specified {@literal field} across a single collection
+		 *
+		 * @param distinctFields name of the field. Must not be {@literal null}.
+		 * @return new instance of {@link ExecutableFindByQuery}.
+		 * @throws IllegalArgumentException if field is {@literal null}.
+		 */
+		FindByQueryWithProjection<T> distinct(String[] distinctFields);
+
+	}
+
+	/**
+	 * {@link ExecutableFindByQuery} provides methods for constructing lookup operations in a fluent way.
+	 */
+
+	interface ExecutableFindByQuery<T> extends FindByQueryWithDistinct<T> {}
 
 }

src/main/java/org/springframework/data/couchbase/core/ExecutableFindByQueryOperationSupport.java

@@ -20,6 +20,7 @@
 
 import org.springframework.data.couchbase.core.ReactiveFindByQueryOperationSupport.ReactiveFindByQuerySupport;
 import org.springframework.data.couchbase.core.query.Query;
+import org.springframework.util.Assert;
 
 import com.couchbase.client.java.query.QueryScanConsistency;
 
@@ -41,28 +42,33 @@ public ExecutableFindByQueryOperationSupport(final CouchbaseTemplate template) {
 
 	@Override
 	public <T> ExecutableFindByQuery<T> findByQuery(final Class<T> domainType) {
-		return new ExecutableFindByQuerySupport<>(template, domainType, ALL_QUERY, QueryScanConsistency.NOT_BOUNDED,
-				"_default._default");
+		return new ExecutableFindByQuerySupport<T>(template, domainType, domainType, ALL_QUERY,
+				QueryScanConsistency.NOT_BOUNDED, null, null);
 	}
 
 	static class ExecutableFindByQuerySupport<T> implements ExecutableFindByQuery<T> {
 
 		private final CouchbaseTemplate template;
-		private final Class<T> domainType;
+		private final Class<?> domainType;
+		private final Class<T> returnType;
 		private final Query query;
 		private final ReactiveFindByQuerySupport<T> reactiveSupport;
 		private final QueryScanConsistency scanConsistency;
 		private final String collection;
+		private final String[] distinctFields;
 
-		ExecutableFindByQuerySupport(final CouchbaseTemplate template, final Class<T> domainType, final Query query,
-				final QueryScanConsistency scanConsistency, final String collection) {
+		ExecutableFindByQuerySupport(final CouchbaseTemplate template, final Class<?> domainType, final Class<T> returnType,
+				final Query query, final QueryScanConsistency scanConsistency, final String collection,
+				final String[] distinctFields) {
 			this.template = template;
 			this.domainType = domainType;
+			this.returnType = returnType;
 			this.query = query;
-			this.reactiveSupport = new ReactiveFindByQuerySupport<T>(template.reactive(), domainType, query, scanConsistency,
-					collection);
+			this.reactiveSupport = new ReactiveFindByQuerySupport<T>(template.reactive(), domainType, returnType, query,
+					scanConsistency, collection, distinctFields);
 			this.scanConsistency = scanConsistency;
 			this.collection = collection;
+			this.distinctFields = distinctFields;
 		}
 
 		@Override
@@ -88,17 +94,44 @@ public TerminatingFindByQuery<T> matching(final Query query) {
 			} else {
 				scanCons = scanConsistency;
 			}
-			return new ExecutableFindByQuerySupport<>(template, domainType, query, scanCons, collection);
+			return new ExecutableFindByQuerySupport<>(template, domainType, returnType, query, scanCons, collection,
+					distinctFields);
 		}
 
 		@Override
-		public FindByQueryConsistentWith<T> consistentWith(final QueryScanConsistency scanConsistency) {
-			return new ExecutableFindByQuerySupport<>(template, domainType, query, scanConsistency, collection);
+		@Deprecated
+		public FindByQueryWithQuery<T> consistentWith(final QueryScanConsistency scanConsistency) {
+			return new ExecutableFindByQuerySupport<>(template, domainType, returnType, query, scanConsistency, collection,
+					distinctFields);
 		}
 
 		@Override
-		public FindByQueryInCollection<T> inCollection(final String collection) {
-			return new ExecutableFindByQuerySupport<>(template, domainType, query, scanConsistency, collection);
+		public FindByQueryConsistentWith<T> withConsistency(final QueryScanConsistency scanConsistency) {
+			return new ExecutableFindByQuerySupport<>(template, domainType, returnType, query, scanConsistency, collection,
+					distinctFields);
+		}
+
+		@Override
+		public FindByQueryWithConsistency<T> inCollection(final String collection) {
+			Assert.hasText(collection, "Collection must not be null nor empty.");
+			return new ExecutableFindByQuerySupport<>(template, domainType, returnType, query, scanConsistency, collection,
+					distinctFields);
+		}
+
+		@Override
+		public <R> FindByQueryInCollection<R> as(Class<R> resturnType) {
+			Assert.notNull(resturnType, "returnType must not be null!");
+
+			return new ExecutableFindByQuerySupport<R>(template, domainType,
+					resturnType, query, scanConsistency, collection,
+					distinctFields);
+		}
+
+		@Override
+		public FindByQueryWithProjection<T> distinct(String[] distinctFields) {
+			Assert.notNull(distinctFields, "distinctFields must not be null!");
+			return new ExecutableFindByQuerySupport<>(template, domainType, returnType, query, scanConsistency, collection,
+					distinctFields);
 		}
 
 		@Override
@@ -115,7 +148,6 @@ public long count() {
 		public boolean exists() {
 			return count() > 0;
 		}
-
 	}
 
 }

src/main/java/org/springframework/data/couchbase/core/ExecutableRemoveByQueryOperation.java

@@ -37,15 +37,23 @@ interface RemoveByQueryWithQuery<T> extends TerminatingRemoveByQuery<T> {
 
 	}
 
+	@Deprecated
 	interface RemoveByQueryConsistentWith<T> extends RemoveByQueryWithQuery<T> {
 
+		@Deprecated
 		RemoveByQueryWithQuery<T> consistentWith(QueryScanConsistency scanConsistency);
 
 	}
 
-	interface RemoveByQueryInCollection<T> extends RemoveByQueryConsistentWith<T> {
+	interface RemoveByQueryWithConsistency<T> extends RemoveByQueryConsistentWith<T> {
 
-		RemoveByQueryConsistentWith<T> inCollection(String collection);
+		RemoveByQueryConsistentWith<T> withConsistency(QueryScanConsistency scanConsistency);
+
+	}
+
+	interface RemoveByQueryInCollection<T> extends RemoveByQueryWithConsistency<T> {
+
+		RemoveByQueryWithConsistency<T> inCollection(String collection);
 
 	}
 

src/main/java/org/springframework/data/couchbase/core/ExecutableRemoveByQueryOperationSupport.java

@@ -21,6 +21,7 @@
 import org.springframework.data.couchbase.core.query.Query;
 
 import com.couchbase.client.java.query.QueryScanConsistency;
+import org.springframework.util.Assert;
 
 public class ExecutableRemoveByQueryOperationSupport implements ExecutableRemoveByQueryOperation {
 
@@ -35,7 +36,7 @@ public ExecutableRemoveByQueryOperationSupport(final CouchbaseTemplate template)
 	@Override
 	public <T> ExecutableRemoveByQuery<T> removeByQuery(Class<T> domainType) {
 		return new ExecutableRemoveByQuerySupport<>(template, domainType, ALL_QUERY, QueryScanConsistency.NOT_BOUNDED,
-				"_default._default");
+				null);
 	}
 
 	static class ExecutableRemoveByQuerySupport<T> implements ExecutableRemoveByQuery<T> {
@@ -69,12 +70,19 @@ public TerminatingRemoveByQuery<T> matching(final Query query) {
 		}
 
 		@Override
+		@Deprecated
 		public RemoveByQueryWithQuery<T> consistentWith(final QueryScanConsistency scanConsistency) {
 			return new ExecutableRemoveByQuerySupport<>(template, domainType, query, scanConsistency, collection);
 		}
 
 		@Override
-		public RemoveByQueryInCollection<T> inCollection(final String collection) {
+		public RemoveByQueryConsistentWith<T> withConsistency(final QueryScanConsistency scanConsistency) {
+			return new ExecutableRemoveByQuerySupport<>(template, domainType, query, scanConsistency, collection);
+		}
+
+		@Override
+		public RemoveByQueryWithConsistency<T> inCollection(final String collection) {
+			Assert.hasText(collection, "Collection must not be null nor empty.");
 			return new ExecutableRemoveByQuerySupport<>(template, domainType, query, scanConsistency, collection);
 		}