Windows not yet supported for torch.compile

[Kilgorio]

(shark.venv) PS E:\shark2\SHARK\shark\examples\shark_inference\llama\llama\llama> python shark_model.py
tensor([[0.4136]])
Traceback (most recent call last):
File "E:\shark2\SHARK\shark\examples\shark_inference\llama\llama\llama\shark_model.py", line 172, in
dynamo_callable = dynamo.optimize(refbackend_torchdynamo_backend)(compile_via_shark)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\shark2\SHARK\shark.venv\Lib\site-packages\torch_dynamo\eval_frame.py", line 479, in optimize
check_if_dynamo_supported()
File "E:\shark2\SHARK\shark.venv\Lib\site-packages\torch_dynamo\eval_frame.py", line 433, in check_if_dynamo_supported
raise RuntimeError("Windows not yet supported for torch.compile")
RuntimeError: Windows not yet supported for torch.compile

idk tbh

[monorimet]

Thanks. I'll take a look.

[powderluv]

Upstream issue is here: pytorch/pytorch#90768

[powderluv]

We could remove the guard and see how far we get on Windows because we don't really use the cpu or triton backends.
We could also just remove the dynamo.optimize() call

[brianmatzelle]

Bypass this on Windows by adding --compile=False to the end of your run statement. Ex:

python train.py config/finetune_conversation.py --compile=False works, whereas
python train.py config/finetune_conversation.py will raise the error

[JohnnyGa]

Adding --compile=False didn't work for me. I've added it just about everywhere I could add it.

[Kilgorio]

Yeah I haven't had this in a while so I guess it's fixed

[cannacak]

Run accelerate config in the command prompt, and when asked, answer the question "Do you wish to optimize your script with Torch Dynamo" with "No."

[d-kleine]

You should be able to turn off compile functionality as this only speeds ups training your model(s).
But this does not resolve the issue. That problem has already been risen some months ago in pytorch/pytorch#90768

Please also see triton-lang/triton#1640