Skip to content

Increase queue length (#27555) #27562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion custom/conf/app.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -1420,7 +1420,7 @@ LEVEL = Info
;DATADIR = queues/ ; Relative paths will be made absolute against `%(APP_DATA_PATH)s`.
;;
;; Default queue length before a channel queue will block
;LENGTH = 100
;LENGTH = 100000
;;
;; Batch size to send for batched queues
;BATCH_LENGTH = 20
Expand Down
2 changes: 1 addition & 1 deletion docs/content/administration/config-cheat-sheet.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -483,7 +483,7 @@ Configuration at `[queue]` will set defaults for queues with overrides for indiv

- `TYPE`: **level**: General queue type, currently support: `level` (uses a LevelDB internally), `channel`, `redis`, `dummy`. Invalid types are treated as `level`.
- `DATADIR`: **queues/common**: Base DataDir for storing level queues. `DATADIR` for individual queues can be set in `queue.name` sections. Relative paths will be made absolute against `%(APP_DATA_PATH)s`.
- `LENGTH`: **100**: Maximal queue size before channel queues block
- `LENGTH`: **100000**: Maximal queue size before channel queues block
- `BATCH_LENGTH`: **20**: Batch data before passing to the handler
- `CONN_STR`: **redis://127.0.0.1:6379/0**: Connection string for the redis queue type. For `redis-cluster` use `redis+cluster://127.0.0.1:6379/0`. Options can be set using query params. Similarly, LevelDB options can also be set using: **leveldb://relative/path?option=value** or **leveldb:///absolute/path?option=value**, and will override `DATADIR`
- `QUEUE_NAME`: **_queue**: The suffix for default redis and disk queue name. Individual queues will default to **`name`**`QUEUE_NAME` but can be overridden in the specific `queue.name` section.
Expand Down
2 changes: 1 addition & 1 deletion docs/content/administration/config-cheat-sheet.zh-cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,7 @@ menu:

- `TYPE`:**level**:通用队列类型,当前支持:`level`(在内部使用 LevelDB)、`channel`、`redis`、`dummy`。无效的类型将视为 `level`。
- `DATADIR`:**queues/common**:用于存储 level 队列的基本 DataDir。单独的队列的 `DATADIR` 可以在 `queue.name` 部分进行设置。相对路径将根据 `%(APP_DATA_PATH)s` 变为绝对路径。
- `LENGTH`:**100**:通道队列阻塞之前的最大队列大小
- `LENGTH`:**100000**:通道队列阻塞之前的最大队列大小
- `BATCH_LENGTH`:**20**:在传递给处理程序之前批处理数据
- `CONN_STR`:**redis://127.0.0.1:6379/0**:redis 队列类型的连接字符串。对于 `redis-cluster`,使用 `redis+cluster://127.0.0.1:6379/0`。可以使用查询参数来设置选项。类似地,LevelDB 选项也可以使用:**leveldb://relative/path?option=value** 或 **leveldb:///absolute/path?option=value** 进行设置,并将覆盖 `DATADIR`。
- `QUEUE_NAME`:**_queue**:默认的 redis 和磁盘队列名称的后缀。单独的队列将默认为 **`name`**`QUEUE_NAME`,但可以在特定的 `queue.name` 部分中进行覆盖。
Expand Down
2 changes: 1 addition & 1 deletion modules/queue/manager_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ CONN_STR = redis://
assert.Equal(t, "default", q.GetName())
assert.Equal(t, "level", q.GetType())
assert.Equal(t, filepath.Join(setting.AppDataPath, "queues/common"), q.baseConfig.DataFullDir)
assert.Equal(t, 100, q.baseConfig.Length)
assert.Equal(t, 100000, q.baseConfig.Length)
assert.Equal(t, 20, q.batchLength)
assert.Equal(t, "", q.baseConfig.ConnStr)
assert.Equal(t, "default_queue", q.baseConfig.QueueFullName)
Expand Down
2 changes: 1 addition & 1 deletion modules/setting/queue.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ func GetQueueSettings(rootCfg ConfigProvider, name string) (QueueSettings, error
queueSettingsDefault := QueueSettings{
Type: "level", // dummy, channel, level, redis
Datadir: "queues/common", // relative to AppDataPath
Length: 100, // queue length before a channel queue will block
Length: 100000, // queue length before a channel queue will block

QueueName: "_queue",
SetName: "_unique",
Expand Down