We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 37a1866 commit baabec4Copy full SHA for baabec4
distributed/tensor_parallelism/fsdp_tp_example.py
@@ -87,7 +87,7 @@
87
dp_mesh = device_mesh["dp"]
88
89
# For TP, input needs to be same across all TP ranks.
90
-# while for SP, input can be different across all ranks.
+# while for DP, input can be different across all ranks.
91
# We will use dp_rank for setting the random seed
92
# to mimic the behavior of the dataloader.
93
dp_rank = dp_mesh.get_local_rank()
0 commit comments