[sharded plugin] Fix check for fp16 precision (#7825)

Co-authored-by: Carlos Mocholí <[email protected]>
Co-authored-by: ananthsub <[email protected]>
(cherry picked from commit ca89a7f)

Author: 3 people

CHANGELOG.md

@@ -208,6 +208,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - Fixed training loop total batch counter when accumulate grad batches was enabled ([#7692](https://github.com/PyTorchLightning/pytorch-lightning/pull/7692))
 
 
+- Fixed a bug where checking `trainer.precision` changed to `'mixed'` when specifying 16 in trainer ([#7825](https://github.com/PyTorchLightning/pytorch-lightning/pull/7825))
+
+
 ## [1.3.2] - 2021-05-18
 
 ### Changed

pytorch_lightning/plugins/training_type/sharded.py

@@ -54,7 +54,8 @@ def _reinit_optimizers_with_oss(self):
                 optim_class = type(optimizer)
                 zero_optimizer = OSS(params=optimizer.param_groups, optim=optim_class, **optimizer.defaults)
                 if _FAIRSCALE_OSS_FP16_BROADCAST_AVAILABLE:
-                    is_fp16 = self.lightning_module.trainer.precision == 16
+                    precision = self.lightning_module.trainer.precision
+                    is_fp16 = precision in ("mixed", 16)
                     # For multi-node training, compressing the model shards in fp16 before broadcasting
                     # improves performance. When using PyTorch AMP, it will not degrade
                     # the model performance.