Research Tarantool for Rivine

[DylanVerstraete]

Currently we are using BoltDB for Rivine, we might want to switch to Tarantool.

[LeeSmet]

@DylanVerstraete please add your findings, e.g. can we indeed embed it in our chain like we can with BoltDB, are there ACID transactions, (No)SQL, etc...

[DylanVerstraete]

Findings:

• Tarantool cannot be embedded into Rivine binary, its a seperate server which you can use Docker for.

• Tarantool stores records in spaces, which are analogous to tables in relational SQL databases. You can have as many as you want — up to 65,000
  You must create spaces upfront by executing Lua based statements.

• Spaces contain tuples, which look like both rows in an SQL table and JSON arrays. By default, maximum tuple size is 1 MB, but it can be changed.

• You must create indexes on spaces upfront by executing Lua based statements. Indexes are required in order to store data, so you must know upfront how each record is going to look like..

  • HASH index: value must be unique and may be arbitrary. This is the principle behind well-known key-value stores, also referred to as maps. A textbook example is MD5 hash (file checksum).

  • TREE index: value may not be unique, but must be “dense”, which enables creating a sorted list. As a result, we get an array that may contain missing values. A good example is the number of an order that gets incremented by one.

• Credentials need to be provided for connecting to Tarantool

• When transactions happen fast it seems like the go-client I am using can't keep up and closes connection to server, I need to investigate this further.

[DylanVerstraete]

Encountered following issue:

tarantool/tarantool#4196

[DylanVerstraete]

Blocked by: tarantool/go-tarantool#80

For parsing data returned from the go-client.

After having a talk with the maintainer following was said by him:

"Yura Sokolov, [9 mei 2019 14:37:46]:
First: you got exactly same answer as in console.

If you use Call17 then there will be one level of “included into slice” lesser.

Second: looks like it is time to implement native SQL protocol command. I don’t know where I’ll have time for cause of personal/family issues. I'll try to do it this month.
At this time you could either parse this generic result, or make a struct with result pattern to get typed result.
I meant “ generic slice of values”. It doesn’t differ from what you get from typical sql connector in dynamic language . For example, python's db2 interface return tuples by default. It is quite same.
"

Which means that current progress is blocked until some native sql protocols are created in this client repo or I create some functions that parse this "generic" data structure but this is not the best solution.