Add support for Couchbase Transactions. #1246

mikereiche · 2021-10-08T03:54:08Z

Closes #1145.

You have read the Spring Data contribution guidelines.
There is a ticket in the bug tracker for the project in our JIRA.
You use the code formatters provided here and have them applied to your changes. Don’t submit any formatting related changes.
You submit test cases (unit or integration tests) that back your changes.
You added yourself as author in the headers of the classes you touched. Amend the date range in the Apache license header if needed. For new types, add the license header (copy from another file and set the current year only).

mikereiche

This is the first draft of the PR.

mikereiche · 2021-10-08T03:57:13Z

src/main/java/org/springframework/data/couchbase/core/ExecutableRemoveByIdOperation.java

-	interface RemoveByIdWithCas extends RemoveByIdWithDurability {
-
+	interface RemoveByIdWithCas extends RemoveByIdWithDurability, WithCas<RemoveResult> {
+		@Override
 		RemoveByIdWithDurability withCas(Long cas);


WithCas() is unrelated. It was missing and I added it at the same time as WithTransaction.

mikereiche · 2021-10-08T04:00:27Z

src/main/java/org/springframework/data/couchbase/core/ReactiveFindByQueryOperationSupport.java

-			return Mono.defer(() -> countResult.onErrorMap(throwable -> {
+			CouchbaseClientFactory clientFactory = template.getCouchbaseClientFactory();
+			ReactiveScope rc = clientFactory.withScope(pArgs.getScope()).getScope().reactive();
+			Mono<ReactiveQueryResult> countResult = null;


all the cluster.query(), collection.query() and ctx.query() return a ReactiveQueryResult, which is nice.

mikereiche · 2021-10-08T04:02:49Z

src/main/java/org/springframework/data/couchbase/core/ReactiveFindByIdOperationSupport.java

+			}
+			return reactiveEntity.onErrorResume(throwable -> {
+				if (throwable instanceof DocumentNotFoundException) {
+					return Mono.empty();
 				}


The kv operations on a collection return a GetResult, while the kv operations on ctx return a TransactionGetResult.

mikereiche · 2021-10-08T04:04:22Z

src/main/java/org/springframework/data/couchbase/core/ReactiveFindByQueryOperationSupport.java

 		}

 		@Override
-		public TerminatingFindByQuery<T> withOptions(final QueryOptions options) {
+		public FindByQueryWithQuery<T> withOptions(final QueryOptions options) {
 			Assert.notNull(options, "Options must not be null.");


This is not related to transactions, but was out of line with how each interface extends the previous one.

mikereiche · 2021-10-08T04:04:58Z

src/main/java/org/springframework/data/couchbase/core/ReactiveFindByQueryOperationSupport.java

 		}

-		public <R> FindByQueryWithConsistency<R> as(Class<R> returnType) {
+		public <R> FindByQueryWithProjecting<R> as(Class<R> returnType) {


This is not related to transactions, but was out of line with how each interface extends the previous one.

mikereiche · 2021-10-08T05:15:33Z

src/test/java/org/springframework/data/couchbase/util/UnmanagedTestCluster.java

@@ -60,7 +60,7 @@ ClusterType type() {

 	@Override
 	TestClusterConfig _start() throws Exception {
-		bucketname = UUID.randomUUID().toString();
+		bucketname = "my_bucket"; // UUID.randomUUID().toString();



To speed up my testing only. Will remove.

mikereiche · 2021-10-08T05:16:43Z

.../org/springframework/data/couchbase/repository/CouchbaseRepositoryQueryIntegrationTests.java

+					.deleteAllById(Arrays.stream(iatas).map((iata) -> "airports::" + iata).collect(Collectors.toSet()));
+		}
+	}
+


An extra test for Group By. Unrelated to transactions.

The fluent operations are common for options that are common to operations with and without transactions. Once there is a transaction(ctx), or an option specific to without-transactions (such as scanConsistency), the interfaces are bifurcated, so that an non-transaction option cannot be applied to a transaction operation, and a transaction(ctx) cannot be applied where a non-transaction option has already been applied. Closes #1145.

Closes 1145.

@transactional

* Porting to SDK-integrated version of transactions The transactions logic exists in the Java SDK as of 3.3.0, with a slightly different API. This is the first effort at the port, which literally just compiles. It will not run as crucial code has been commented and todo-ed. There is work remaining to figure out how to complete the port, as some crucial parts (such as ctx.commit() and ctx.rollback()) have been intentionally removed. * Continuing work to get the ExtSDKIntegration port working Trying to transition to CallbackPreferring manager. * Added CouchbaseSimpleCallbackTransactionManager, the simplest possible implementation of CallbackPreferringTransactionManager, combined with a simpler approach to ThreadLocal storage in ReactiveInsertByIdSupport. Test 'commitShouldPersistTxEntriesOfTxAnnotatedMethod' is now passing. * Adding WIP get-and-replace @transactional support (Not yet working as CAS/version field in Person is not populated correctly.)

…g-projects/spring-data-couchbase into datacouch_1145_transaction_support

…nup.

@transactional

* Porting to SDK-integrated version of transactions The transactions logic exists in the Java SDK as of 3.3.0, with a slightly different API. This is the first effort at the port, which literally just compiles. It will not run as crucial code has been commented and todo-ed. There is work remaining to figure out how to complete the port, as some crucial parts (such as ctx.commit() and ctx.rollback()) have been intentionally removed. * Continuing work to get the ExtSDKIntegration port working Trying to transition to CallbackPreferring manager. * Added CouchbaseSimpleCallbackTransactionManager, the simplest possible implementation of CallbackPreferringTransactionManager, combined with a simpler approach to ThreadLocal storage in ReactiveInsertByIdSupport. Test 'commitShouldPersistTxEntriesOfTxAnnotatedMethod' is now passing. * Adding WIP get-and-replace @transactional support (Not yet working as CAS/version field in Person is not populated correctly.) * Transitioning to use CoreTransactionAttemptContext. Tests may fail. * Removing AttemptContextReactiveAccessor Don't think we need this, as we can pass around CoreTransactionAttemptContext instead, which gives access to a lot of internals. * Removing TransactionsReactive Would prefer not to C&P a huge class out of the transaction internals, and don't think we need it. * Removing some files not currently used To reduce & simplify the amount of code to look at. Some don't seem to be used in any branch, some just aren't used in this branch. * Removing CouchbaseTransactionInterceptor As per offline discussion, CallbackPreferringPlatformTransactionManager is perhaps the optimal solution. * Copying @transactional tests out into separate class * Tidyup * Tidyup test names * Verify GenericSupport is on same thread before and after transactional operation * Refactoring CouchbaseSimpleCallbackTransactionManager ThreadLocalStorage management * Using latest java-client * ReactiveReplaceByIdSupport - Fixing use of CAS now have CoreTransactionAttemptContext. Removing unused code. * ReactiveInsertByIdSupport - fixing use of reactive vs non-reactive, and CAS * Merging upstream * Remove incorrect thread check (.doOnNext could execute on a different thread) * Completing merge from upstream * Removing unused classes * Give GenericSupport a better name * Reject at runtime options that aren't supported in a transaction * Fixing some small todos, partly by removing unused coe * Fix runtime option checks * Simplifying CouchbaseSimpleCallbackTransactionManager ThreadLocalStorage Standardising on ReactiveCouchbaseResourceHolder rather than holding CoreTransactionAttemptContext too * Removing version from CouchbaseDocument Can't recall why I added this, and tests pass without it * Improving CouchbaseTransactionalIntegrationTests and adding more tests * Reject operations that aren't allowed in a transaction (upsertById etc.) * Improve handling of CAS mismatch By calling CoreTransactionAttemptContext.operationFailed, it ensures that internal state is set. So even if the user catches the exception, the transaction still behaves as it should. * Removing a now-redundant non-transactional check on upsertById I missed this when adding TransactionalSupport.verifyNotInTransaction here. * Support @transactional options timeout and isolation level Co-authored-by: Graham Pople <[email protected]>

mikereiche requested a review from daschl October 8, 2021 03:54

mikereiche force-pushed the datacouch_1145_transaction_support branch from 3422a20 to d294b2f Compare October 8, 2021 05:07

mikereiche commented Oct 8, 2021

View reviewed changes

mikereiche force-pushed the datacouch_1145_transaction_support branch from d294b2f to b53bbde Compare October 8, 2021 05:21

mikereiche force-pushed the datacouch_1145_transaction_support branch 3 times, most recently from 25fcb14 to 1679651 Compare October 22, 2021 00:06

mikereiche force-pushed the datacouch_1145_transaction_support branch from a1a2a1d to 3a33f44 Compare November 16, 2021 03:19

mikereiche force-pushed the datacouch_1145_transaction_support branch 2 times, most recently from 33d8b5f to 778b10e Compare February 23, 2022 19:12

mikereiche force-pushed the datacouch_1145_transaction_support branch from fa8ee88 to a226f88 Compare March 15, 2022 04:04

mikereiche force-pushed the datacouch_1145_transaction_support branch from a226f88 to 7b1ba51 Compare April 11, 2022 22:13

Support for @transactional for blocking and reactive Transactionmanager.

b4a93d7

Closes 1145.

mikereiche force-pushed the datacouch_1145_transaction_support branch from 7b1ba51 to b4a93d7 Compare April 11, 2022 23:08

Transaction Support.

0165fdb

mikereiche force-pushed the datacouch_1145_transaction_support branch 3 times, most recently from 2e6d152 to fde39ed Compare April 26, 2022 18:08

Transaction Support.

95bf2fc

mikereiche force-pushed the datacouch_1145_transaction_support branch from fde39ed to 95bf2fc Compare April 26, 2022 21:59

programmatix and others added 3 commits May 3, 2022 11:22

Commit before pulling Graham's changes.

4e93e55

Merge branch 'datacouch_1145_transaction_support' of github.com:sprin…

56ad3fa

…g-projects/spring-data-couchbase into datacouch_1145_transaction_support

mikereiche force-pushed the datacouch_1145_transaction_support branch 2 times, most recently from df19157 to fe82566 Compare May 24, 2022 13:37

mikereiche added 2 commits May 24, 2022 12:45

Get scope and collection from pseudoArgs and some cleanup.

bc4dc01

Add ReactiveTransactionWrapper/TransactionWrapper and a bunch of clea…

9e7fdde

…nup.

mikereiche force-pushed the datacouch_1145_transaction_support branch from fe82566 to 9e7fdde Compare May 26, 2022 21:50

mikereiche closed this Jun 17, 2022

mikereiche deleted the datacouch_1145_transaction_support branch July 28, 2022 22:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Couchbase Transactions. #1246

Add support for Couchbase Transactions. #1246

mikereiche commented Oct 8, 2021

mikereiche left a comment

mikereiche Oct 8, 2021

mikereiche Oct 8, 2021

mikereiche Oct 8, 2021

mikereiche Oct 8, 2021

mikereiche Oct 8, 2021

mikereiche Oct 8, 2021

mikereiche Oct 8, 2021

Add support for Couchbase Transactions. #1246

Add support for Couchbase Transactions. #1246

Conversation

mikereiche commented Oct 8, 2021

mikereiche left a comment

Choose a reason for hiding this comment

mikereiche Oct 8, 2021

Choose a reason for hiding this comment

mikereiche Oct 8, 2021

Choose a reason for hiding this comment

mikereiche Oct 8, 2021

Choose a reason for hiding this comment

mikereiche Oct 8, 2021

Choose a reason for hiding this comment

mikereiche Oct 8, 2021

Choose a reason for hiding this comment

mikereiche Oct 8, 2021

Choose a reason for hiding this comment

mikereiche Oct 8, 2021

Choose a reason for hiding this comment