Fix for cannot import name 'LlamaTokenizerFast' (#647) (#648)

Co-authored-by: jagane-infinstor <[email protected]>
Co-authored-by: Ammar Ahmad Awan <[email protected]>

Author: 3 people

inference/huggingface/text-generation/README.md

@@ -15,6 +15,13 @@ Python dependencies:
 <pre>
 pip install -r requirements.txt
 </pre>
+If you are using conda, the following works:
+<pre>
+conda create -c conda-forge -n deepspeed python=3.10
+conda activate deepspeed
+pip install -r requirements.txt
+deepspeed --num_gpus 1 inference-test.py --name bigscience/bloom-3b --batch_size 2
+</pre>
 
 # Inference Test
 

inference/huggingface/text-generation/requirements.txt

@@ -1,3 +1,3 @@
 deepspeed
 torch
-transformers==4.21.2
+transformers==4.28.1