Skip to content

Commit 8613b60

Browse files
mwrmwr
authored
Merge pull request #10 from run-as-root/feature/#3-module-documentation
Module documentation
2 parents 7acdbd1 + 2fc66f5 commit 8613b60

File tree

12 files changed

+210
-17
lines changed

12 files changed

+210
-17
lines changed

‎LICENCE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 run-as-root
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NON INFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 181 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,181 @@
1-
Magento 2 message queue retry
1+
# run-as-root/magento2-message-queue-retry
2+
3+
It gives the possibility to process the same queue message more than once,
4+
utilizing The RabbitMQ's [dead letter exchange](https://www.rabbitmq.com/dlx.html) feature.
5+
6+
## Table of Contents
7+
- [Prerequisites](#prerequisites)
8+
- [Installation](#installation)
9+
- [Features](#features)
10+
- [How it works](#how-it-works)
11+
- [Configuration](#configuration)
12+
- [License](#licence)
13+
14+
## Prerequisites
15+
16+
- Magento 2.4.5 or higher
17+
- PHP 8.1 or higher
18+
- RabbitMQ
19+
20+
To be able to use this module, you have to manually configure the dead letter exchange(s) for the queue(s) you want to enable the retry mechanismm through the `queue_topology.xml` file.
21+
An example will be given in the [Configuration](#configuration) section.
22+
23+
Other requisite is that your exchanges have to have a relation from one exchange to only one topic and queue,
24+
25+
For example:
26+
27+
![topology](docs/queue-requirement.png)
28+
29+
## Installation
30+
31+
To install the module via composer:
32+
```bash
33+
composer require run-as-root/magento2-message-queue-retry
34+
```
35+
36+
To enable the module:
37+
```bash
38+
bin/magento module:enable RunAsRoot_MessageQueueRetry
39+
```
40+
41+
## Features
42+
43+
- Toggle activation
44+
- Configure the retry limit for a queue
45+
- Admin grid to manage the messages with the retry limit reached
46+
- Requeue the failed messages to their origin queue
47+
- Delete the message
48+
- Download the message body
49+
50+
## How it works
51+
52+
The default Magento's consumer behavior is to reject the message when an exception is thrown during the consumer's execution.
53+
If you use a standard configuration for the queue (without a dead-letter exchange), the message will be discarded and not processed again.
54+
55+
This behavior will change a bit with this module. It will introduce an extra step that will check if the message has reached your retry limit,
56+
if so, it will be discarded from RabbitMQ and sent to the `run_as_root_message` Mysql table and stay there until manual management through the admin panel.
57+
58+
If the message has not reached the retry limit, it will be rejected, and RabbitMQ will send it to the dead letter exchange. The message will be routed automatically to the "delay" queue and stay there until de TTL time is reached.
59+
After the TTL time is reached, the message will be returned to its original queue.
60+
61+
The diagram below illustrates both approaches:
62+
63+
![img.png](docs/flow.png)
64+
65+
In the admin panel a new grid will be available to manage the messages that have reached the retry limit:
66+
67+
Path: RunAsRoot > Manage Messages
68+
69+
![img.png](docs/messages-grid.png)
70+
71+
The grid colums:
72+
73+
- **Topic name**: The name of the topic that the message belongs to
74+
- **Total retries**: The total number of times the message has been processed
75+
- **Failure Description**: The exception message that was thrown during the processing of the message
76+
- **Message Body**: The original message body, it can be downloaded as a file and it is in JSON format.
77+
78+
The grid actions:
79+
80+
- **Requeue**: It will send the message to the original queue
81+
- **Download**: It will download the message body content as a JSON file
82+
83+
The grid also has a mass action to delete or requeue the selected messages.
84+
85+
Is possible to configure the ACL for each action in the grid and the module configuration:
86+
87+
![img.png](docs/acl.png)
88+
89+
### Configuration
90+
91+
Two steps are necessary to configure the retry for a queue:
92+
1. Configure the dead letter exchange
93+
1. Enable the message queue retry and delclare the retry limit configuration
94+
95+
Let's imagine a scenario that the `erp_order_export` queue already exists in your project and to simplify the example the topic name, exchange name and queue name are the same: `erp_order_export`.
96+
97+
We need to change these two files in order to declare and configure the delay queue:
98+
- `communication.xml`
99+
- `queue_topology.xml`
100+
101+
**The current queue configuration are like this**:
102+
103+
`etc/communication.xml`:
104+
105+
```xml
106+
<?xml version="1.0"?>
107+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
108+
xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
109+
<topic name="erp_order_export" request="string"/>
110+
</config>
111+
```
112+
113+
`etc/queue_topology.xml`:
114+
115+
```xml
116+
<?xml version="1.0"?>
117+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
118+
xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
119+
<exchange name="erp_order_export" connection="amqp" type="topic">
120+
<binding id="erp_order_export" topic="erp_order_export" destinationType="queue" destination="erp_order_export"/>
121+
</exchange>
122+
</config>
123+
```
124+
125+
**You have to change it to**:
126+
127+
`etc/communication.xml`:
128+
129+
```xml
130+
<?xml version="1.0"?>
131+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
132+
xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
133+
<topic name="erp_order_export" request="string"/>
134+
<topic name="erp_order_export_delay" request="string"/>
135+
</config>
136+
```
137+
138+
`etc/queue_topology.xml`:
139+
```xml
140+
<?xml version="1.0"?>
141+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
142+
xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
143+
<exchange name="erp_order_export" connection="amqp" type="topic">
144+
<binding id="erp_order_export" topic="erp_order_export" destinationType="queue" destination="erp_order_export">
145+
<arguments>
146+
<argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export_delay</argument>
147+
<argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export_delay</argument>
148+
</arguments>
149+
</binding>
150+
</exchange>
151+
<!-- Delay queue -->
152+
<exchange name="erp_order_export_delay" connection="amqp" type="topic">
153+
<binding id="erp_order_export_delay" topic="erp_order_export_delay" destinationType="queue" destination="erp_order_export_delay">
154+
<arguments>
155+
<argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export</argument>
156+
<argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export</argument>
157+
<argument name="x-message-ttl" xsi:type="number">300000</argument>
158+
</arguments>
159+
</binding>
160+
</exchange>
161+
</config>
162+
```
163+
164+
In the `erp_order_export` exchange binding, we added the `x-dead-letter-exchange` and `x-dead-letter-routing-key` arguments, this will route the message to the `erp_order_export_delay` exchange when the message is rejected.
165+
166+
We added the `erp_order_export_delay` exchange and binding, it points to the original exchange (`erp_order_export`). the `x-message-ttl` argument will configure the period that the message will stay in the `erp_order_export_delay` queue, in this example for 5 minutes (300000ms). When the lifetime expires (TTL), RabbitMQ will send the message to `erp_order_export` automatically.
167+
168+
The `erp_order_export_delay` queue does not have a consumer, it will be used only to hold(delay) messages according with the period defined in the `x-message-ttl` argument.
169+
170+
Now you have to define toggle the activation for the retry queue module and declare the retry limit for the queue:
171+
172+
System > Configuration > RUN-AS-ROOT > Message Queue Retry
173+
174+
![img.png](docs/configuration.png)
175+
176+
Note that if the queue is not declared in the configuration it will the default Magento consumer behavior.
177+
178+
For more information of how to configure message queues in Magento 2, you can take a look [here](https://developer.adobe.com/commerce/php/development/components/message-queues/configuration/).
179+
180+
## License
181+
[MIT](https://opensource.org/licenses/MIT)

‎docs/acl.png‎

22.3 KB
Loading

‎docs/configuration.png‎

117 KB
Loading

‎docs/flow.png‎

57.1 KB
Loading

‎docs/messages-grid.png‎

72.7 KB
Loading

‎docs/queue-requirement.png‎

62.1 KB
Loading

‎src/Block/Adminhtml/QueuesConfig.php‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,6 @@ protected function _prepareToRender(): void
1616
[ 'label' => __('Main Topic Name'), 'class' => 'required-entry' ]
1717
);
1818

19-
$this->addColumn(
20-
MessageQueueRetryConfig::DELAY_TOPIC_NAME,
21-
[ 'label' => __('Delay Topic Name'), 'class' => 'required-entry' ]
22-
);
23-
2419
$this->addColumn(
2520
MessageQueueRetryConfig::RETRY_LIMIT,
2621
[ 'label' => __('Retry Limit'), 'class' => 'required-entry validate-zero-or-greater' ]

‎src/Ui/Component/Listing/Columns/MessageBody.php‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,10 @@ public function prepareDataSource(array $dataSource): array
1616
continue;
1717
}
1818

19+
if (mb_strlen($item['message_body']) < 100) {
20+
continue;
21+
}
22+
1923
$item['message_body'] = substr($item['message_body'], 0, 100) . '...';
2024
}
2125
}

‎src/etc/adminhtml/system.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
<frontend_model>RunAsRoot\MessageQueueRetry\Block\Adminhtml\QueuesConfig</frontend_model>
2929
<backend_model>RunAsRoot\MessageQueueRetry\Model\Config\Backend\QueuesConfig</backend_model>
3030
<comment>
31-
Please, provide the topic name of the delay queue that is within the communication.xml file.
31+
Please, provide the topic name of the queue that is within the communication.xml file.
3232
After the retry limit is reached the message will be sent to the failed messages grid.
3333
</comment>
3434
</field>

0 commit comments

Comments
 (0)