Skip to content

Commit fddc220

Browse files
committed
feat: Add docs
Resolves #2671
1 parent 199067f commit fddc220

File tree

3 files changed

+253
-0
lines changed

3 files changed

+253
-0
lines changed
Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
---
2+
title: "Microservices Transactional Outbox Pattern in Java: Ensuring Reliable Messaging"
3+
shortTitle: Transactional Outbox
4+
description: "Learn how the Transactional Outbox pattern guarantees reliable message delivery between microservices by leveraging a local database transaction, achieving eventual consistency."
5+
category: Integration
6+
language: en
7+
tag:
8+
- Microservices
9+
- Messaging
10+
- Fault tolerance
11+
- Decoupling
12+
- Data consistency
13+
- Enterprise patterns
14+
---
15+
16+
## Also known as
17+
18+
* Outbox Pattern
19+
* Reliable Messaging Pattern
20+
21+
## Intent of Microservices Transactional Outbox Design Pattern
22+
23+
To ensure that messages are reliably sent from a microservice as part of a single, atomic database transaction, preventing data loss and inconsistencies in distributed systems.
24+
25+
## Detailed Explanation of Microservices Transactional Outbox Pattern with Real-World Examples
26+
27+
Real-world example
28+
> Imagine an e-commerce platform's "Order Service." When a new order is placed, the service must save the order to its database and also notify a separate "Notification Service" to send a confirmation email. If the Order Service first saves the order and then tries to publish a message, the message broker could be down, resulting in an order being created without a notification. Conversely, if it sends the message first and then the database commit fails, a notification is sent for an order that doesn't exist. The Transactional Outbox pattern solves this by saving the new order and the "email notification" event into an `outbox` table within the same database transaction. A separate process then reads from this `outbox` table and reliably sends the event to the Notification Service, guaranteeing that a notification is sent if, and only if, the order was successfully created.
29+
30+
In plain words
31+
> Atomically save your business data and the messages about those changes in your local database before sending them to other services.
32+
33+
Chris Richardson's "microservices.io" says
34+
> The Transactional Outbox pattern ensures that a message is sent if and only if the database transaction that creates the event commits. The service that sends the message has an "outbox" table in its database. When it sends a message, it inserts the message into the outbox table as part of the same transaction that updates its business entities. A separate message relay process reads the outbox table and publishes the messages to a message broker.
35+
36+
Flowchart
37+
38+
![Microservices Transactional Outbox flowchart](./etc/microservices-transactional-outbox-flowchart.png)
39+
40+
## Programmatic Example of Microservices Transactional Outbox Pattern in Java
41+
42+
This example demonstrates the Transactional Outbox pattern for a `CustomerService`. When a new customer is created, the business data is saved, and a corresponding event is stored in an `outbox` table within the same transaction. A background poller then reads these events and sends them to a message broker.
43+
44+
The `OutboxEvent` entity represents a record in our `outbox` table.
45+
46+
```java
47+
@Entity
48+
@Table(name = "OUTBOX")
49+
public class OutboxEvent {
50+
51+
@Id
52+
@GeneratedValue
53+
private Integer id;
54+
55+
private String eventType;
56+
private String payload; // Typically a JSON string
57+
private boolean processed;
58+
private LocalDateTime createdAt;
59+
60+
// Constructors, Getters, and Setters
61+
}
62+
```
63+
64+
The `CustomerService` handles the business logic. It saves a new `Customer` and an `OutboxEvent` in a single, atomic database transaction.
65+
66+
```java
67+
public class CustomerService {
68+
69+
private final EntityManager entityManager;
70+
private final OutboxRepository outboxRepository;
71+
72+
public void createCustomer(String username) throws Exception {
73+
entityManager.getTransaction().begin();
74+
try {
75+
// 1. Save the business entity
76+
var customer = new Customer(username);
77+
entityManager.persist(customer);
78+
79+
// 2. Create and save the outbox event in the same transaction
80+
String payload = new ObjectMapper().writeValueAsString(customer);
81+
var event = new OutboxEvent("CUSTOMER_CREATED", payload);
82+
outboxRepository.save(event);
83+
84+
// 3. Commit the single transaction
85+
entityManager.getTransaction().commit();
86+
} catch (Exception e) {
87+
entityManager.getTransaction().rollback();
88+
throw e;
89+
}
90+
}
91+
}
92+
```
93+
94+
The `EventPoller` acts as the separate process that reads from the outbox and publishes messages.
95+
96+
```java
97+
public class EventPoller {
98+
99+
private final EntityManager entityManager;
100+
private final OutboxRepository outboxRepository;
101+
private final MessageBroker messageBroker;
102+
103+
public void start() {
104+
// Polls the database at a fixed rate
105+
}
106+
107+
private void processOutboxEvents() {
108+
entityManager.getTransaction().begin();
109+
try {
110+
List<OutboxEvent> events = outboxRepository.findUnprocessedEvents();
111+
for (var event : events) {
112+
messageBroker.sendMessage(event);
113+
outboxRepository.markAsProcessed(event);
114+
}
115+
entityManager.getTransaction().commit();
116+
} catch (Exception e) {
117+
entityManager.getTransaction().rollback();
118+
}
119+
}
120+
}
121+
```
122+
123+
The main application starts the services and simulates customer creation.
124+
125+
```java
126+
public class App {
127+
128+
public static void main(String[] args) throws Exception {
129+
var entityManagerFactory = Persistence.createEntityManagerFactory("transactional-outbox-pu");
130+
var entityManager = entityManagerFactory.createEntityManager();
131+
132+
var customerService = new CustomerService(entityManager);
133+
var messageBroker = new MessageBroker();
134+
var eventPoller = new EventPoller(entityManager, messageBroker);
135+
136+
// Start the background poller
137+
eventPoller.start();
138+
139+
// Simulate application logic
140+
customerService.createCustomer("john.doe");
141+
142+
// Shutdown
143+
eventPoller.stop();
144+
}
145+
}
146+
```
147+
## When to Use the Microservices Transactional Outbox Pattern in Java
148+
149+
* When you need to guarantee that an event or message is published after a database transaction successfully commits.
150+
* In distributed systems where you need to reliably communicate state changes between services.
151+
* When using asynchronous communication patterns to improve resilience and decoupling but cannot afford to lose messages.
152+
* To avoid dual-write problems where a service needs to write to its own database and send a message as a single atomic operation.
153+
154+
## Real-World Applications of Microservices Transactional Outbox Pattern in Java
155+
156+
* E-commerce platforms for reliably handling order creation, payment confirmation, and shipping notification events.
157+
* Financial systems for ensuring that transaction notifications and audit logs are created and sent reliably.
158+
* Booking and reservation systems where a confirmed booking must trigger reliable notifications to other systems (e.g., inventory, customer communication)
159+
160+
## Benefits and Trade-offs of Microservices Transactional Outbox Pattern
161+
162+
Benefits:
163+
164+
* `Reliability`: Guarantees at-least-once delivery of messages, as the event is persisted within the same transaction as the business data.
165+
* `Data Consistency`: Prevents inconsistencies between a service's internal state and the messages it sends to other services.
166+
* `Decoupling`: The service's business logic is completely decoupled from the complexities of message publishing, retries, and failure handling.
167+
168+
Trade-offs:
169+
170+
* `Increased Complexity`: Requires an additional `outbox` database table and a separate message relay/polling process.
171+
* `Latency`: Messages are not sent in real-time. There is a delay between the transaction commit and the message being published by the poller.
172+
* `Potential` for Duplicate Messages: Because it ensures at-least-once delivery, consumers of the messages must be designed to be idempotent to handle potential duplicates.
173+
174+
## Related Java Design Patterns
175+
176+
* `Saga Pattern`: The Transactional Outbox pattern is a common and reliable way to implement the steps in a Saga, ensuring that commands or events are published reliably between saga participants.
177+
* `Publish/Subscribe`: The outbox poller typically publishes messages to a topic on a message broker, which are then consumed by one or more subscribers.
178+
* `Event Sourcing`: While different, both patterns involve persisting state changes as a sequence of events. The outbox pattern can be used to reliably publish events generated in an Event Sourcing system.
179+
180+
## References and Credits
181+
182+
* [Pattern: Transactional Outbox (microservices.io)](https://microservices.io/patterns/data/transactional-outbox.html)
183+
* [Outbox Pattern for Microservices Architectures](https://medium.com/design-microservices-architecture-with-patterns/outbox-pattern-for-microservices-architectures-1b8648dfaa27)
184+
* [Outbox Pattern in Microservices](https://www.baeldung.com/cs/outbox-pattern-microservices)
32.7 KB
Loading
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
@startuml
2+
title Transactional Outbox Pattern Class Diagram
3+
4+
package com.iluwatar.transactionaloutbox {
5+
6+
class App {
7+
+ {static} main(args: String[]): void
8+
}
9+
10+
class Customer {
11+
- id: Integer
12+
- username: String
13+
+ Customer(username: String)
14+
}
15+
16+
class OutboxEvent {
17+
- id: Integer
18+
- eventType: String
19+
- payload: String
20+
- processed: boolean
21+
+ OutboxEvent(eventType: String, payload: String)
22+
}
23+
24+
class CustomerService {
25+
- entityManager: EntityManager
26+
- outboxRepository: OutboxRepository
27+
+ CustomerService(entityManager: EntityManager)
28+
+ createCustomer(username: String): void
29+
}
30+
31+
class OutboxRepository {
32+
- entityManager: EntityManager
33+
+ OutboxRepository(entityManager: EntityManager)
34+
+ save(event: OutboxEvent): void
35+
+ markAsProcessed(event: OutboxEvent): void
36+
+ findUnprocessedEvents(): List<OutboxEvent>
37+
}
38+
39+
class EventPoller {
40+
- outboxRepository: OutboxRepository
41+
- messageBroker: MessageBroker
42+
+ EventPoller(entityManager: EntityManager, messageBroker: MessageBroker)
43+
+ start(): void
44+
+ stop(): void
45+
- processOutboxEvents(): void
46+
}
47+
48+
class MessageBroker {
49+
+ sendMessage(event: OutboxEvent): void
50+
}
51+
}
52+
53+
' --- Relationships ---
54+
55+
App ..> CustomerService : creates >
56+
App ..> EventPoller : creates >
57+
App ..> MessageBroker : creates >
58+
59+
CustomerService --> "-outboxRepository" OutboxRepository
60+
CustomerService ..> Customer : <<creates>>
61+
CustomerService ..> OutboxEvent : <<creates>>
62+
63+
EventPoller --> "-outboxRepository" OutboxRepository
64+
EventPoller --> "-messageBroker" MessageBroker
65+
66+
OutboxRepository ..> OutboxEvent : <<manages>>
67+
MessageBroker ..> OutboxEvent : <<sends>>
68+
69+
@enduml

0 commit comments

Comments
 (0)