Fairseq v0.10.2 compatible

[JulianneKnott]

Updated generate.py to be compatible with Fairseq v0.10.2.
Speed tbd.

[JiushengChen]

Avoid checking in those txt files and old backup file - "generate_old.py".

[feihugis]

Thanks @JulianneKnott for this PR! I did a pass and have one question for the files under results: are they used for comparing the outputs from fastseq and fairseq? It will be very helpful if we can add a unit test to automatically check it. We can also integrate it with our CI in the next step.

[JulianneKnott]

Thanks @JulianneKnott for this PR! I did a pass and have one question for the files under results: are they used for comparing the outputs from fastseq and fairseq? It will be very helpful if we can add a unit test to automatically check it. We can also integrate it with our CI in the next step.

Yes. Didn't mean to commit them and will remove them on the next commit. Will look into adding a unit test also.

[JulianneKnott]

Benchmark Info:

Util Model Task Split BatchSize Samples Tokens Bleu Rouge Loss Perplexity Runtime(seconds) Throughput(samples/s) Throughput(tokens/s)
fairseq_v0.10.2 bart.large.cnn cnn_dm/len-1024.bin valid 32 13367 1098036 17.93 NA NA NA 3875 3.4 283.4
fairseq_v0.10.2+fastseq_v0.0.4 bart.large.cnn cnn_dm/len-1024.bin valid 32 13367 1098036 17.93 NA NA NA 3028 4.4 362.6

[feihugis]

The current code looks good to me. As this change will break the other parts of the main branch, could we create another branch (e.g., fairseq-0.10.2) for this work? After finishing all the tasks, it can be merged to the main branch.

[JulianneKnott]

Updated beam search optimizer.
Benchmarks:

Util Model Task Split BatchSize Samples Tokens Bleu Rouge Loss Perplexity Runtime(seconds) Throughput(samples/s) Throughput(tokens/s)
fairseq_v0.10.2+fastseq_v0.0.4 bart.large.cnn cnn_dm/len-1024.bin valid 256 13367 1098278 17.90 NA NA NA 718 18.6 1529.6

[JulianneKnott]

Updated el attention optimizer
Benchmarks:

Util Model Task Split BatchSize Samples Tokens Bleu Rouge Loss Perplexity Runtime(seconds) Throughput(samples/s) Throughput(tokens/s)
fairseq_v0.10.2+fastseq_v0.0.4 bart.large.cnn cnn_dm/len-1024.bin valid 320 13367 1098535 17.92 NA NA NA 621 21.5 1769.0

[feihugis]

Thanks @JulianneKnott ! LGTM. Just find some very minor issues. After fixing, the PR will be good to merge.

[feihugis]

I'm not sure if people will get confused when the child class has the same name with the parent class? What do @JulianneKnott think of it?

[yuyan2do]

Looks good. Just added a few suggestions inline.

Another question: Is there any test cover converting a model to its TorchScript version? If not, we need do it manually to verify our change do not break this functionality.

[yuyan2do]

Where is this param used?

[JulianneKnott]

fastseq/fastseq/optimizer/fairseq/generate.py

Line 152 in 4a3f17e

if args.decode_hypothesis:

also lines: 211, 218, 240 in the same file