Skip to content

Commit 8fee0a3

Browse files
committed
Implementation of batch replace
Batch upsert is mostly used for operation with one bucket / one Tarantool node in a transaction. In this case batch replace is more efficient then replacing tuple-by-tuple. Right now CRUD cannot provide batch replace with full consistency. CRUD offers batch upsert with partial consistency. That means that full consistency can be provided only on single replicaset using `box` transactions. Part of #193
1 parent f2caaf3 commit 8fee0a3

File tree

9 files changed

+2557
-0
lines changed

9 files changed

+2557
-0
lines changed

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
88
## [Unreleased]
99

1010
### Added
11+
* Batch insert/upsert operation
12+
`crud.insert_many()`/`crud.insert_object_many()`/
13+
`crud.upsert_many()`/`crud.upsert_object_many()`
14+
`crud.replace_many()`/`crud.replace_object_many()`
15+
with partial consistency
1116

1217
### Changed
1318

README.md

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ It also provides the `crud-storage` and `crud-router` roles for
2121
- [Update](#update)
2222
- [Delete](#delete)
2323
- [Replace](#replace)
24+
- [Replace many](#replace-many)
2425
- [Upsert](#upsert)
2526
- [Upsert many](#upsert-many)
2627
- [Select](#select)
@@ -477,6 +478,101 @@ crud.replace_object('customers', {
477478
...
478479
```
479480

481+
### Replace many
482+
483+
```lua
484+
-- Replace batch of tuples
485+
local result, err = crud.replace_many(space_name, tuples, opts)
486+
-- Replace batch of objects
487+
local result, err = crud.replace_object_many(space_name, objects, opts)
488+
```
489+
490+
where:
491+
492+
* `space_name` (`string`) - name of the space to insert/replace an object
493+
* `tuples` / `objects` (`table`) - array of tuples/objects to insert
494+
* `opts`:
495+
* `timeout` (`?number`) - `vshard.call` timeout (in seconds)
496+
* `fields` (`?table`) - field names for getting only a subset of fields
497+
* `stop_on_error` (`?boolean`) - stop on a first error and report error
498+
regarding the failed operation and error about what tuples were not
499+
performed, default is `false`
500+
* `rollback_on_error` (`?boolean`) - any failed operation will lead to
501+
rollback on a storage, where the operation is failed, report error
502+
about what tuples were rollback, default is `false`
503+
504+
Returns metadata and array contains inserted rows, array of errors.
505+
Error object can contain field `operation_data`.
506+
507+
This field can contain:
508+
* tuple for which the error occurred;
509+
* object with an incorrect format;
510+
* tuple the operation on which was performed but
511+
operation was rollback;
512+
* tuple the operation on which was not performed
513+
because operation was stopped by error.
514+
515+
Right now CRUD cannot provide batch replace with full consistency.
516+
CRUD offers batch replace with partial consistency. That means
517+
that full consistency can be provided only on single replicaset
518+
using `box` transactions.
519+
520+
**Example:**
521+
522+
```lua
523+
crud.replace_many('developers', {
524+
{1, box.NULL, 'Elizabeth', 'lizaaa'},
525+
{2, box.NULL, 'Anastasia', 'iamnewdeveloper'},
526+
})
527+
---
528+
- metadata:
529+
- {'name': 'id', 'type': 'unsigned'}
530+
- {'name': 'bucket_id', 'type': 'unsigned'}
531+
- {'name': 'name', 'type': 'string'}
532+
- {'name': 'login', 'type': 'string'}
533+
rows:
534+
- [1, 477, 'Elizabeth', 'lizaaa']
535+
- [2, 401, 'Anastasia', 'iamnewdeveloper']
536+
...
537+
crud.replace_object_many('developers', {
538+
{id = 1, name = 'Inga', login = 'mylogin'},
539+
{id = 10, name = 'Anastasia', login = 'qwerty'},
540+
})
541+
---
542+
- metadata:
543+
- {'name': 'id', 'type': 'unsigned'}
544+
- {'name': 'bucket_id', 'type': 'unsigned'}
545+
- {'name': 'name', 'type': 'string'}
546+
- {'name': 'age', 'type': 'number'}
547+
rows:
548+
- [1, 477, 'Inga', 'mylogin']
549+
- [10, 569, 'Anastasia', 'qwerty']
550+
551+
-- Partial success
552+
-- Let's say login has unique secondary index
553+
local res, errs = crud.replace_object_many('developers', {
554+
{id = 22, name = 'Alex', login = 'pushkinn'},
555+
{id = 3, name = 'Anastasia', login = 'qwerty'},
556+
{id = 5, name = 'Sergey', login = 's.petrenko'},
557+
})
558+
---
559+
res
560+
- metadata:
561+
- {'name': 'id', 'type': 'unsigned'}
562+
- {'name': 'bucket_id', 'type': 'unsigned'}
563+
- {'name': 'name', 'type': 'string'}
564+
- {'name': 'age', 'type': 'number'}
565+
rows:
566+
- [5, 1172, 'Sergey', 'crudisthebest'],
567+
- [22, 655, 'Alex', 'pushkinn'],
568+
569+
#errs -- 1
570+
errs[1].class_name -- ReplaceManyError
571+
errs[1].err -- 'Duplicate key exists <...>'
572+
errs[1].tuple -- {3, 2804, 'Anastasia', 'qwerty'}
573+
...
574+
```
575+
480576
### Upsert
481577

482578
```lua

crud.lua

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ local cfg = require('crud.cfg')
66
local insert = require('crud.insert')
77
local insert_many = require('crud.insert_many')
88
local replace = require('crud.replace')
9+
local replace_many = require('crud.replace_many')
910
local get = require('crud.get')
1011
local update = require('crud.update')
1112
local upsert = require('crud.upsert')
@@ -53,6 +54,14 @@ crud.replace = stats.wrap(replace.tuple, stats.op.REPLACE)
5354
-- @function replace_object
5455
crud.replace_object = stats.wrap(replace.object, stats.op.REPLACE)
5556

57+
-- @refer replace_many.tuples
58+
-- @function replace_many
59+
crud.replace_many = replace_many.tuples
60+
61+
-- @refer replace_many.objects
62+
-- @function replace_object_many
63+
crud.replace_object_many = replace_many.objects
64+
5665
-- @refer update.call
5766
-- @function update
5867
crud.update = stats.wrap(update.call, stats.op.UPDATE)
@@ -145,6 +154,7 @@ function crud.init_storage()
145154
insert_many.init()
146155
get.init()
147156
replace.init()
157+
replace_many.init()
148158
update.init()
149159
upsert.init()
150160
upsert_many.init()

0 commit comments

Comments
 (0)