Cascading delete for Part

[eywalker]

The cascading delete still doesn't handle the case involving dj.Part correctly. Basically, any deletion on dj.Part should cause deletion on its parent table. Marking deletion on the parent table then should cause appropriate cascading of deletion on the parent's as well as Part's subtables.

[dimitri-yatsenko]

The term "subtable", as we have sometimes used it, is now replaced with "part relation". Let's also use "master relation" to indicate the relation containing the part relation. We have also decided that part relations cannot have their own part relations.

So to restate the present issue:

• Deleting from a part relation should not be possible without deleting matching tuples from its master relation.

We have two options to address this issue:

1. prohibit deletes from a part relation that are not cascaded through its master relation
2. delete from the master relation when a delete is issued on the part relation

Option 1 seems overly restrictive since a part relation may have dependencies not shared by its master and some cascading deletes may halt and force the user to break up a single delete command into several more specific commands. If we pursue this solution, we need to make sure that the error message and the suggested course of action are clear.

Option 2 seems a bit aggressive since now deletes will cascade one level up the dependency hierarchy as well as down. This may be counterintuitive to some users. This concern is partly addressed by displaying a summary of the data to be deleted before committing the delete. Also option 2 may produce more complex queries that may be much slower than ones resulting from Option 1.

Which option should we choose?

[eywalker]

I almost always prefer strong and convenient (but potentially destructive) solution, provided that we keep it informative. So if I have to choose, I'd prefer to use option 2. However, I also think this is a perfect opportunity for us to use config to alter between different behaviors, and offer both option 1 and option 2.

[fabiansinz]

I prefer option 2 and I think we should offer to change the behavior in the config.

[dimitri-yatsenko]

I support Option 2. However, making datajoint configurable between these two options would create undue documentation and testing burden. Let's implement Option 2 and document it well.

[dimitri-yatsenko]

I relabeled this as a 'feature' because the current implementation acts reasonably: it fails to delete when attempting to delete from parts, bypassing the master. I just don't want panic to set in that cascading deletes are unsafe.

[mspacek]

Hello! OK, so the term "subtable" is no longer used. It's crucial to update the documentation to reflect that. I just spent several hours trying to figure out how to define a subtable, including searching through the code, and finally ended up here.

Am I correct in assuming the original issue reported above by @eywalker remains? I'm trying to define a part relation, such that if I try to delete parts, I'm either not allowed to, or I am but it automatically deletes the appropriate tuples in the parent (or "master part"). At the moment, I'm not seeing either option 1 or 2 behaviour. I've followed the example I found in datajoint.tests.schema.Ephys.Channel. Maybe the easiest thing to do for now is to define a delete() method that does nothing? I've added this to my part relation:

        def delete(self):
            """Override self.delete() so that it does nothing other than print a message"""
            print("direct deletion from %s is disabled because it's a part relation "
                  "(subtable) of %r" % (self.__class__, self._master))

and it seems to work as intended.

[dimitri-yatsenko]

Hi @mspacek, the master-part relationship has now been defined consistently between matlab and python and we can move quickly on the documentation. Thank you for your patience.

Yes, this issue is still open. The master-part relationship has not been strongly enforced. Now you are able to delete from part without deleting from master. This should not be done and eventually we need to enforce the master-part discipline by either Option 1 or Option 2. I like Option 1 because it is easy to implement correctly. Option 2 is more involved but may be more user friendly.

[dimitri-yatsenko]

I propose to implement Option 1 now similar to what @mspacek did. If we run into problems, we will create a new issue to implement Option 2.