Skip to content

Commit b58b1ee

Browse files
oleg-jukovecoleg-jukovec
committed
msgpack: add optional msgpack.v5 support
The msgpack.v5 code located in msgpack_v5.go and msgpack_v5_helper_test.go for tests. It is the same logic for submodules. An user can use msgpack.v5 with a build tag: go_tarantool_msgpack_v5 Part of #124
1 parent 36b43bb commit b58b1ee

16 files changed

+301
-17
lines changed

‎CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ CI and documentation.
3131
- queue-utube handling (#85)
3232
- Master discovery (#113)
3333
- SQL support (#62)
34+
- Optional msgpack.v5 usage (#124)
3435

3536
### Changed
3637

‎README.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ faster than other packages according to public benchmarks.
2626
* [Documentation](#documentation)
2727
* [API reference](#api-reference)
2828
* [Walking\-through example](#walking-through-example)
29+
* [msgpack.v5 migration](#msgpackv5-migration)
2930
* [Contributing](#contributing)
3031
* [Alternative connectors](#alternative-connectors)
3132

@@ -59,6 +60,15 @@ To disable SSL support and linking with OpenSSL, you can use the tag:
5960
go_tarantool_ssl_disable
6061
```
6162

63+
To replace usage of `msgpack.v2` with `msgpack.v5` you need to use the build
64+
tag:
65+
```
66+
go_tarantool_msgpack_v5
67+
```
68+
**Note:** In future releases, `msgpack.v5` may be used by default. We recommend
69+
to read [msgpack.v5 migration notes](#msgpackv5-migration) and try to
70+
use msgpack.v5 before the changes.
71+
6272
## Documentation
6373

6474
Read the [Tarantool documentation][tarantool-doc-data-model-url]
@@ -129,6 +139,30 @@ There are two parameters:
129139
* a space number (it could just as easily have been a space name), and
130140
* a tuple.
131141

142+
### msgpack.v5 migration
143+
144+
Most function names and argument types in `msgpack.v5` and `msgpack.v2`
145+
have not changed (except at least `EncodeInt`, `EncodeUint` and `RegisterExt`).
146+
But there are a lot of changes in a logic of encoding and deconding. On the plus
147+
side the migration seems easy, but on the minus side you need to be very
148+
careful.
149+
150+
First of all, `EncodeInt8`, `EncodeInt16`, `EncodeInt32`, `EncodeInt64`
151+
and `EncodeUint*` analogues at `msgpack.v5` encode numbers as is without loss of
152+
type. In `msgpack.v2` the type of a number is reduced to a value.
153+
154+
Secondly, a base decoding function does not convert numbers to `int64` or
155+
`uint64`. The change makes manual type conversions much more difficult and can
156+
lead to runtime erros with an old code. We recommend not use type conversions
157+
and give preference to `*Typed` functions (besides, it's faster).
158+
159+
There are also changes in the logic that can lead to errors in the old code,
160+
[as example](https://github.com/vmihailenco/msgpack/issues/327). Although in
161+
`msgpack.v5` has added some functions for the logic tuning see
162+
[UseLooseInterfaceDecoding](https://pkg.go.dev/github.com/vmihailenco/msgpack/v5#Decoder.UseLooseInterfaceDecoding), [UseCompactInts](https://pkg.go.dev/github.com/vmihailenco/msgpack/v5#Encoder.UseCompactInts) etc, but it is still impossible
163+
to achieve full compliance of behavior between `msgpack.v5` and `msgpack.v2`. So
164+
we don't go this way. We use standart settings if it possible.
165+
132166
## Contributing
133167

134168
See [the contributing guide](CONTRIBUTING.md) for detailed instructions on how

‎go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,5 @@ require (
1010
google.golang.org/appengine v1.6.7 // indirect
1111
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
1212
gopkg.in/vmihailenco/msgpack.v2 v2.9.2
13+
github.com/vmihailenco/msgpack/v5 v5.3.5
1314
)

‎go.sum

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,15 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
1616
github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572 h1:RC6RW7j+1+HkWaX/Yh71Ee5ZHaHYt7ZP4sQgUrm6cDU=
1717
github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572/go.mod h1:w0SWMsp6j9O/dk4/ZpIhL+3CkG8ofA2vuv7k+ltqUMc=
1818
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
19+
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
1920
github.com/stretchr/testify v1.7.1 h1:5TQK59W5E3v0r2duFAb7P95B6hEeOyEnHRa8MjYSMTY=
2021
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
2122
github.com/tarantool/go-openssl v0.0.8-0.20220419150948-be4921aa2f87 h1:JGzuBxNBq5saVtPUcuu5Y4+kbJON6H02//OT+RNqGts=
2223
github.com/tarantool/go-openssl v0.0.8-0.20220419150948-be4921aa2f87/go.mod h1:M7H4xYSbzqpW/ZRBMyH0eyqQBsnhAMfsYk5mv0yid7A=
24+
github.com/vmihailenco/msgpack/v5 v5.3.5 h1:5gO0H1iULLWGhs2H5tbAHIZTV8/cYafcFOr9znI5mJU=
25+
github.com/vmihailenco/msgpack/v5 v5.3.5/go.mod h1:7xyJ9e+0+9SaZT0Wt1RGleJXzli6Q/V5KbhBonMG9jc=
26+
github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
27+
github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
2328
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
2429
golang.org/x/net v0.0.0-20190603091049-60506f45cf65 h1:+rhAzEzT3f4JtomfC371qB+0Ola2caSKcY69NUBZrRQ=
2530
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=

‎msgpack.go

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package tarantool
25

36
import (
@@ -54,53 +57,53 @@ func msgpackIsString(code byte) bool {
5457
}
5558

5659
func (s *single) DecodeMsgpack(d *msgpack.Decoder) error {
57-
return s.decodeMsgpackImpl(&Decoder{d})
60+
return s.decodeMsgpackImpl(&Decoder{Decoder: d})
5861
}
5962

6063
func (space *Space) DecodeMsgpack(d *msgpack.Decoder) error {
61-
return space.decodeMsgpackImpl(&Decoder{d})
64+
return space.decodeMsgpackImpl(&Decoder{Decoder: d})
6265
}
6366

6467
func (field *Field) DecodeMsgpack(d *msgpack.Decoder) error {
65-
return field.decodeMsgpackImpl(&Decoder{d})
68+
return field.decodeMsgpackImpl(&Decoder{Decoder: d})
6669
}
6770

6871
func (index *Index) DecodeMsgpack(d *msgpack.Decoder) error {
69-
return index.decodeMsgpackImpl(&Decoder{d})
72+
return index.decodeMsgpackImpl(&Decoder{Decoder: d})
7073
}
7174

7275
func (indexField *IndexField) DecodeMsgpack(d *msgpack.Decoder) error {
73-
return indexField.decodeMsgpackImpl(&Decoder{d})
76+
return indexField.decodeMsgpackImpl(&Decoder{Decoder: d})
7477
}
7578

7679
func (meta *ColumnMetaData) DecodeMsgpack(d *msgpack.Decoder) error {
77-
return meta.decodeMsgpackImpl(&Decoder{d})
80+
return meta.decodeMsgpackImpl(&Decoder{Decoder: d})
7881
}
7982

8083
func (info *SQLInfo) DecodeMsgpack(d *msgpack.Decoder) error {
81-
return info.decodeMsgpackImpl(&Decoder{d})
84+
return info.decodeMsgpackImpl(&Decoder{Decoder: d})
8285
}
8386

8487
func (k IntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
85-
return k.encodeMsgpackImpl(&Encoder{enc})
88+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
8689
}
8790

8891
func (k UintKey) EncodeMsgpack(enc *msgpack.Encoder) error {
89-
return k.encodeMsgpackImpl(&Encoder{enc})
92+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
9093
}
9194

9295
func (k StringKey) EncodeMsgpack(enc *msgpack.Encoder) error {
93-
return k.encodeMsgpackImpl(&Encoder{enc})
96+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
9497
}
9598

9699
func (k IntIntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
97-
return k.encodeMsgpackImpl(&Encoder{enc})
100+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
98101
}
99102

100103
func (o Op) EncodeMsgpack(enc *msgpack.Encoder) error {
101-
return o.encodeMsgpackImpl(&Encoder{enc})
104+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
102105
}
103106

104107
func (o OpSplice) EncodeMsgpack(enc *msgpack.Encoder) error {
105-
return o.encodeMsgpackImpl(&Encoder{enc})
108+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
106109
}

‎msgpack_helper_test.go

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package tarantool_test
25

36
import (
@@ -6,17 +9,17 @@ import (
69
)
710

811
func (m *Member) EncodeMsgpack(e *msgpack.Encoder) error {
9-
return m.encodeMsgpackImpl(&Encoder{e})
12+
return m.encodeMsgpackImpl(&Encoder{Encoder: e})
1013
}
1114

1215
func (m *Member) DecodeMsgpack(d *msgpack.Decoder) error {
13-
return m.decodeMsgpackImpl(&Decoder{d})
16+
return m.decodeMsgpackImpl(&Decoder{Decoder: d})
1417
}
1518

1619
func (c *Tuple2) EncodeMsgpack(e *msgpack.Encoder) error {
17-
return c.encodeMsgpackImpl(&Encoder{e})
20+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
1821
}
1922

2023
func (c *Tuple2) DecodeMsgpack(d *msgpack.Decoder) error {
21-
return c.decodeMsgpackImpl(&Decoder{d})
24+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
2225
}

‎msgpack_v5.go

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package tarantool
5+
6+
import (
7+
"io"
8+
9+
"github.com/vmihailenco/msgpack/v5"
10+
"github.com/vmihailenco/msgpack/v5/msgpcode"
11+
)
12+
13+
type Encoder struct {
14+
*msgpack.Encoder
15+
}
16+
17+
type Decoder struct {
18+
*msgpack.Decoder
19+
}
20+
21+
func newEncoder(w io.Writer) *Encoder {
22+
enc := msgpack.NewEncoder(w)
23+
return &Encoder{Encoder: enc}
24+
}
25+
26+
func newDecoder(r io.Reader) *Decoder {
27+
dec := msgpack.NewDecoder(r)
28+
dec.SetMapDecoder(func(dec *msgpack.Decoder) (interface{}, error) {
29+
return dec.DecodeUntypedMap()
30+
})
31+
return &Decoder{Decoder: dec}
32+
}
33+
34+
func (e *Encoder) encodeUintImpl(v uint64) error {
35+
return e.EncodeUint(v)
36+
}
37+
38+
func (e *Encoder) encodeIntImpl(v int64) error {
39+
return e.EncodeInt(v)
40+
}
41+
42+
func msgpackIsUint(code byte) bool {
43+
return code == msgpcode.Uint8 || code == msgpcode.Uint16 ||
44+
code == msgpcode.Uint32 || code == msgpcode.Uint64 ||
45+
msgpcode.IsFixedNum(code)
46+
}
47+
48+
func msgpackIsMap(code byte) bool {
49+
return code == msgpcode.Map16 || code == msgpcode.Map32 || msgpcode.IsFixedMap(code)
50+
}
51+
52+
func msgpackIsArray(code byte) bool {
53+
return code == msgpcode.Array16 || code == msgpcode.Array32 ||
54+
msgpcode.IsFixedArray(code)
55+
}
56+
57+
func msgpackIsString(code byte) bool {
58+
return msgpcode.IsFixedString(code) || code == msgpcode.Str8 ||
59+
code == msgpcode.Str16 || code == msgpcode.Str32
60+
}
61+
62+
func (s *single) DecodeMsgpack(d *msgpack.Decoder) error {
63+
return s.decodeMsgpackImpl(&Decoder{Decoder: d})
64+
}
65+
66+
func (space *Space) DecodeMsgpack(d *msgpack.Decoder) error {
67+
return space.decodeMsgpackImpl(&Decoder{Decoder: d})
68+
}
69+
70+
func (field *Field) DecodeMsgpack(d *msgpack.Decoder) error {
71+
return field.decodeMsgpackImpl(&Decoder{Decoder: d})
72+
}
73+
74+
func (index *Index) DecodeMsgpack(d *msgpack.Decoder) error {
75+
return index.decodeMsgpackImpl(&Decoder{Decoder: d})
76+
}
77+
78+
func (indexField *IndexField) DecodeMsgpack(d *msgpack.Decoder) error {
79+
return indexField.decodeMsgpackImpl(&Decoder{Decoder: d})
80+
}
81+
82+
func (meta *ColumnMetaData) DecodeMsgpack(d *msgpack.Decoder) error {
83+
return meta.decodeMsgpackImpl(&Decoder{Decoder: d})
84+
}
85+
86+
func (info *SQLInfo) DecodeMsgpack(d *msgpack.Decoder) error {
87+
return info.decodeMsgpackImpl(&Decoder{Decoder: d})
88+
}
89+
90+
func (k IntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
91+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
92+
}
93+
94+
func (k UintKey) EncodeMsgpack(enc *msgpack.Encoder) error {
95+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
96+
}
97+
98+
func (k StringKey) EncodeMsgpack(enc *msgpack.Encoder) error {
99+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
100+
}
101+
102+
func (k IntIntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
103+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
104+
}
105+
106+
func (o Op) EncodeMsgpack(enc *msgpack.Encoder) error {
107+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
108+
}
109+
110+
func (o OpSplice) EncodeMsgpack(enc *msgpack.Encoder) error {
111+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
112+
}

‎msgpack_v5_helper_test.go

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package tarantool_test
5+
6+
import (
7+
. "github.com/tarantool/go-tarantool"
8+
"github.com/vmihailenco/msgpack/v5"
9+
)
10+
11+
func (m *Member) EncodeMsgpack(e *msgpack.Encoder) error {
12+
return m.encodeMsgpackImpl(&Encoder{Encoder: e})
13+
}
14+
15+
func (m *Member) DecodeMsgpack(d *msgpack.Decoder) error {
16+
return m.decodeMsgpackImpl(&Decoder{Decoder: d})
17+
}
18+
19+
func (c *Tuple2) EncodeMsgpack(e *msgpack.Encoder) error {
20+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
21+
}
22+
23+
func (c *Tuple2) DecodeMsgpack(d *msgpack.Decoder) error {
24+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
25+
}

‎queue/msgpack.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package queue
25

36
import (

‎queue/msgpack_helper_test.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package queue_test
25

36
import (

‎queue/msgpack_v5.go

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package queue
5+
6+
import (
7+
"github.com/tarantool/go-tarantool"
8+
"github.com/vmihailenco/msgpack/v5"
9+
)
10+
11+
func (qd *queueData) DecodeMsgpack(d *msgpack.Decoder) error {
12+
return qd.decodeMsgpackImpl(&tarantool.Decoder{Decoder: d})
13+
}
14+
15+
func (t *Task) DecodeMsgpack(d *msgpack.Decoder) error {
16+
return t.decodeMsgpackImpl(&tarantool.Decoder{Decoder: d})
17+
}

‎queue/msgpack_v5_helper_test.go

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package queue_test
5+
6+
import (
7+
. "github.com/tarantool/go-tarantool"
8+
"github.com/vmihailenco/msgpack/v5"
9+
)
10+
11+
func (c *customData) DecodeMsgpack(d *msgpack.Decoder) error {
12+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
13+
}
14+
15+
func (c *customData) EncodeMsgpack(e *msgpack.Encoder) error {
16+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
17+
}
18+
19+
func (c *dummyData) DecodeMsgpack(d *msgpack.Decoder) error {
20+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
21+
}
22+
23+
func (c *dummyData) EncodeMsgpack(e *msgpack.Encoder) error {
24+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
25+
}

‎uuid/msgpack.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package uuid
25

36
import (

‎uuid/msgpack_helper_test.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package uuid_test
25

36
import (

0 commit comments

Comments
 (0)