Skip to content

Commit 0944ff7

Browse files
committed
Fix memory leak related to MerginProject object (fixes #78)
It turns out that we are creating MerginProject objects way too much, and the cleanup of it may not be handled properly because of a cyclic reference between MerginProject and its internal GeoDiff context object. It is going to be both safer and faster to create MerginProject for a particular working directory once and then keep it cached. Fun fact: the leak is not showing up in "top" commend, only in docker's stats about memory...
1 parent 11ce231 commit 0944ff7

File tree

1 file changed

+23
-4
lines changed

1 file changed

+23
-4
lines changed

dbsync.py

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -173,9 +173,28 @@ def _print_mergin_changes(diff_dict):
173173
print(" removed: " + item['path'])
174174

175175

176+
# Dictionary used by _get_mergin_project() function below.
177+
# key = path to a local dir with Mergin project, value = cached MerginProject object
178+
cached_mergin_project_objects = {}
179+
180+
181+
def _get_mergin_project(work_path):
182+
"""
183+
Returns a cached MerginProject object or creates one if it does not exist yet.
184+
This is to avoid creating many of these objects (e.g. every pull/push) because it does
185+
initialization of geodiff as well, so things should be 1. a bit faster, and 2. safer.
186+
(Safer because we are having a cycle of refs between GeoDiff and MerginProject objects
187+
related to logging - and untangling those would need some extra calls when we are done
188+
with MerginProject. But since we use the object all the time, it's better to cache it anyway.)
189+
"""
190+
if work_path not in cached_mergin_project_objects:
191+
cached_mergin_project_objects[work_path] = MerginProject(work_path)
192+
return cached_mergin_project_objects[work_path]
193+
194+
176195
def _get_project_version(work_path):
177196
""" Returns the current version of the project """
178-
mp = MerginProject(work_path)
197+
mp = _get_mergin_project(work_path)
179198
return mp.metadata["version"]
180199

181200

@@ -234,7 +253,7 @@ def pull(conn_cfg, mc):
234253
_check_has_working_dir(work_dir)
235254
_check_has_sync_file(gpkg_full_path)
236255

237-
mp = MerginProject(work_dir)
256+
mp = _get_mergin_project(work_dir)
238257
mp.set_tables_to_skip(ignored_tables)
239258
if mp.geodiff is None:
240259
raise DbSyncError("Mergin Maps client installation problem: geodiff not available")
@@ -322,7 +341,7 @@ def status(conn_cfg, mc):
322341
_check_has_sync_file(gpkg_full_path)
323342

324343
# get basic information
325-
mp = MerginProject(work_dir)
344+
mp = _get_mergin_project(work_dir)
326345
mp.set_tables_to_skip(ignored_tables)
327346
if mp.geodiff is None:
328347
raise DbSyncError("Mergin Maps client installation problem: geodiff not available")
@@ -394,7 +413,7 @@ def push(conn_cfg, mc):
394413
_check_has_working_dir(work_dir)
395414
_check_has_sync_file(gpkg_full_path)
396415

397-
mp = MerginProject(work_dir)
416+
mp = _get_mergin_project(work_dir)
398417
mp.set_tables_to_skip(ignored_tables)
399418
if mp.geodiff is None:
400419
raise DbSyncError("Mergin Maps client installation problem: geodiff not available")

0 commit comments

Comments
 (0)