On thread safety #18
Description
This is an observation from prototyping a storage system similar in semantics to TinyDB's ConcurrencyMiddleware- note that the current approach of TinyDB, although it's thread-safe, writes may have a chance of being discarded. Consider a hypothetical situation where I have a table with the ConcurrencyMiddleware being used, i.e.:
s = ConcurrencyMiddleware(JSONStorage('file'))
db = TinyDB('data.json', storage=s).table('table')
If we spawn say, three threads, though it is guaranteed by the lock that their writes will be atomic, and their reads atomic as well, some writes may be discarded due to the way TinyDB works, i.e. in the insert function:
def insert(self, element):
"""
Insert a new element into the table.
"""
current_id = self._last_id + 1
self._last_id = current_id
data = self._read()
data[current_id] = element
self._write(data)
Setting the key on a new dictionary (reconstructed every time the _read() function is called) is, though atomic, there may be multiple threads calling this function asynchronously and unless they happen to execute database calls serially, there is no way to guarantee that the previous write is acknowledged when the data is written to disk- that depends on which write request writes to the file last, a.k.a. race condition. Solving this problem requires the lock to be held on the entirety of the function call, i.e.
def insert(self, element):
with self.lock:
...