[RFC] Introduce `Strategy` in favor of `TrainingTypePlugin`

[kaushikb11]

Proposed refactor

Part of #9932
Related PR: #10548

Motivation

As we are introducing StrategyPlugin in favor of TrainingTypePlugin for this release.

@ananthsub raised some really good points for discussion regarding StrategyPlugin. Reference

Proposed solution

Three main points to be discussed here

• Drop Plugin from the name of the class
• It should be under pytorch_lightning/strategy directory
• Over time, these classes should be renamed as DDPStrategy and so on.

Would like your thoughts? @PyTorchLightning/core-contributors

cc @Borda @justusschock @awaelchli @akihironitta @kaushikb11 @ananthsub

[awaelchli]

Yes, definitely. We agreed on this I think. This was already set in motion by the introduction of the strategy argument in the Trainer.

Over time, these classes should be renamed as DDPStrategy and so on.

Has to be done for 1.6. We are targeting stable API for these components at 1.6.

[awaelchli]

Should we name all classes without the "Plugin"/"Strategy" suffix? Here is how this would look like:

DataParallel, (or just DP)
DDP2,
DDP,
DDPSpawn,
DDPFullySharded,
DeepSpeed,
Horovod,
IPU,
SingleDevice,
SingleTPU,
TPUSpawn,
TrainingType,
Parallel,
DDPSharded,
DDPSpawnSharded,

@ananthsub @kaushikb11 @carmocca @justusschock

[ananthsub]

On naming, Loggers, Profilers, Accelerators, and ClusterEnvironments are suffixed with the component name (e.g. TensorboardLogger, SimpleProfiler, GPUAccelerator, SlurmEnvironment).

If we drop the suffix for strategy:

• Would we also make this change for other components of Lightning so the naming scheme is unified across components (leading to one less thing to think about for users) ?
• Do you think there's a risk with naming collisions? For instance, the IPU accelerator and IPU strategy could both be IPU classes. As a result, users would likely need to change the import for them to be:

from pytorch_lightning.accelerators import IPU as IPUAccelerator
from pytorch_lightning.strategy import IPU as IPUStrategy