You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[💬 DeepSpeed Chat and DeepSpeed Community 💬](#-deepspeed-chat-and-deepspeed-community-)
@@ -387,6 +388,33 @@ model family | size range
387
388
388
389
* All performance and accuracy tests have been performed using the OPT model family only. For other models, please see our training_scripts folder on how to change model families.
A DeepSpeed CI workflow runs the DeepSpeed-Chat Step 3 pipeline nightly across the following test configurations:
399
+
400
+
Models
401
+
```
402
+
Actor: facebook/opt-125m
403
+
Critic: facebook/opt-125m (trained in DS-Chat Step 2)
404
+
```
405
+
406
+
Parameters comprising test matrix
407
+
```
408
+
Zero Stage: 2, 3
409
+
Hybrid Engine: True, False
410
+
Offload: True, False
411
+
LoRA: True, False
412
+
```
413
+
414
+
Each configuration (16 total) runs through a limited number of Step 3 non-overflow training steps (i.e. steps where neither actor nor critic overflow) and saves the actor/critic models.
415
+
Assertions are used to check if the training pipeline executed correctly and if the actor and critic models were saved properly.
416
+
417
+
390
418
## ⚓ Documentation and Tutorial ⚓
391
419
392
420
For more APIs, example scripts, and evaluation results, please refer to
0 commit comments