Detokenization parallelization public repo #69
Conversation
|
|
CNN 1k data set is too small now, result is not reliable. Please use full valid set. |
|
|
For |
This is because pytorch dataloader API only supports number of samples to be multiple of batch size. This is why last batch is dropped. |
|
@JiushengChen For Mbart and T5 , do we have larger dataset? I couldn't find it in benchmark scripts. |
The root cause may be here. Try to change it to be |
Yes, I have larger data in my local. Please leave out these two, I will update them today. |
I already did. |
Could you please explain more? I saw your code here use |
benchmarks/models/hf_distibart.sh
Outdated
| grep -E "transformers_v3.0.2\+fastseq_v.* hf.sshleifer.distilbart-cnn-12-6.tar.gz cnn_dm.1k/raw val 64 " perf | awk '{s+=$13}END{print s/NR}' | bash range.sh 13 100 | ||
| grep -E "transformers_v3.0.2\+fastseq_v.* hf.sshleifer.distilbart-cnn-12-6.tar.gz cnn_dm.1k/raw val 64 " perf | awk '{s+=$13}END{print s/NR}' | bash range.sh 15.2 100 | ||
| # todo: bigger bs doesn't increase speed | ||
| grep -E "transformers_v3.0.2\+fastseq_v.* hf.sshleifer.distilbart-cnn-12-6.tar.gz cnn_dm.1k/raw val 128 " perf | awk '{s+=$13}END{print s/NR}' | bash range.sh 13.5 100 | ||
| grep -E "transformers_v3.0.2\+fastseq_v.* hf.sshleifer.distilbart-cnn-12-6.tar.gz cnn_dm.1k/raw val 128 " perf | awk '{s+=$13}END{print s/NR}' | bash range.sh 15.9 100 |
|
Benchmarks on Larger dataset:
|
|
Before/After
|
Synced offline. Included last batch. |
3 participants
Moving #37 here.