[MNT] reimplement horizontal/vertical slicing and remove __finalize__ from _constructor(_sliced)

[theOehrly]

Pandas is in the progress of deprecating to allow non-class returns from _constructor and _constructor_sliced (pandas-dev/pandas#52420).

This in turn means that FastF1 needs to change its implementations for differentiating between horizontal and vertical slicing of dataframes.
Also, the __finalize__ call that is done as a patch to work around the incomplete implementation in pandas needs to be removed from the constructor properties.

[jorisvandenbossche]

To be clear, there is not agreement at all that we want to do this, and as such there are no concrete plans to actually deprecate this.

The actual discussion is at pandas-dev/pandas#51772 (the mentioned PR is a bit premature).

This in turn means that FastF1 needs to change its implementations for differentiating between horizontal and vertical slicing of dataframes.

I am curious about this, can you point to the relevant code? (it's useful for us to better understand real-world subclassing use cases)
Looking at for example the Telemetry subclass, the _constructor function only seems to do the extra __finalize__ call (which is a good workaround for pandas failing to call it in some places, as you mention)

[theOehrly]

I know that there is no agreement on this yet. But I want to keep a close eye on this and therefore have opened this issue as a kind of reminder. If Pandas deprecates the callable constructor (and _metadata as well), important parts of how internal structures in FastF1 currently work will fall apart and need to be reimplemented in some way.

There are two ways in which FastF1 currently uses the callable constructor:

• __finalize__ in the constructor as a catch-all workaround for incomplete metadata propagation (at a small performance cost).
  __finalize__ would need to be called explicitly after every problematic operation.
• horizontal and vertical slicing of dataframes, here is an example:

  Fast-F1/fastf1/core.py

  Lines 2943 to 2953 in f5446af

  	def _constructor_sliced(self):
  	def _new(*args, **kwargs):
  	name = kwargs.get('name')
  	if name and (name in self.columns):
  	# vertical slice
  	return pd.Series(*args, **kwargs).__finalize__(self)
  	# horizontal slice
  	return DriverResult(*args, **kwargs).__finalize__(self)
  	return _new

  
  This could be implemented in __new__ as well, I guess, but only because the column names for basically all DataFrame-like objects in FastF1 are basically constant. So the reference to self.columns is not necessary. But it would then fail, as soon as users add their own columns to a DataFrame (which they can do).

Deprecating _metadata would be more problematic than deprecating the callable constructor, but I see this one happening sooner, potentially.

[jorisvandenbossche]

Thanks for the explanation, that's super useful!

Deprecating _metadata would be more problematic than deprecating the callable constructor, but I see this one happening sooner, potentially.

To be honest, I don't think this will happen any time soon. We also make use of this in GeoPandas, without easy ways around it (I am a maintainer of both pandas and GeoPandas)

For GeoPandas it is a similar situation, I think: the ability to have a custom new function in _constructor is mostly convenient, but can be workaround somewhat (but at the same time I don't really see the benefit for pandas to actually disallow this). But _metadata is much more fundamental, and is a large part of what makes subclassing useful/working.