[pc] migrate progression futures from list to deque

[bobrenjc93]

Stack from ghstack (oldest at bottom):

• [wip] async cancellation test #157652
• [pc] verify max autotune is in generated source code #157650
• [pc] introduce ProgressiveCompilationState and clear callback #157619
• -> [pc] migrate progression futures from list to deque #157614

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/157614

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit be05e6c with merge base c808af5 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• trunk / macos-py3-arm64 / test (default, 1, 3, macos-m1-stable) (gh) (disabled by #148644 but the issue was closed recently and a rebase is needed to make it pass)
  dynamo/test_decorators.py::DecoratorTests::test_set_stance_aot_eager_then_compile

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

• pull / cuda12.8-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu, unstable) (gh) (#153987)
  MISSING REGRESSION TEST

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[aorenste]

nit: Above (github won't let me comment on unchanged lines):

    progression_futures: Sequence[Future[_WireProtocolPickledOutput]],

[bobrenjc93]

Problem is then mypy complains about no popleft

  Error (MYPY) [attr-defined]
    "Sequence[Future[_WireProtocolPickledOutput]]" has no attribute "popleft"

        268  |
        269  |        # Clear earlier progression futures to free memory
        270  |        for _ in range(stage_index + 1):
    >>> 271  |            self._progression_futures.popleft()
        272  |
        273  |    @override
        274  |    def post_compile(

[aorenste]

I think you misunderstood. I meant the init signature should take Sequence. The instance member should still be deque. (I can see that what I wrote wasn't clear)

[pytorchmergebot]

Starting merge as part of PR stack under #157650