Skip to content

Conversation

@stratika
Copy link
Collaborator

@stratika stratika commented Jun 5, 2025

This PR provides support and instructions to install and run on Windows OS, as discussed in #10. The changes included in this PR are listed below:

  • README, update the installation part for Windows and removed unnecessary steps.
  • Add a set_paths.cmd file for loading environment variables on Windows
  • Updated the llama-tornado python program with: i) a fix for the path delimiter in Windows, and ii) loading the variables before launching the execution.

This is the trace after running on an NVIDIA GPU in Windows (no isolation mode):

python llama-tornado --gpu --verbose-init --opencl --model beehive-llama-3.2-1b-instruct-fp16.gguf --prompt "Tell me a joke"
WARNING: Using incubator modules: jdk.incubator.vector
Parse beehive-llama-3.2-1b-instruct-fp16.gguf: 1427 millis
Loading model weights in TornadoVM format (loading F16 -> F16)
Load LlaMa model: 2911 millis
Starting TornadoVM initialization...
TornadoVM GPU execution plan creation: 3117.03 ms
Java to GPU JIT compiler warmup: 3383.45 ms
Transfer read-only weights to GPU: 1043.90 ms
Finished TornadoVM initialization...

I'll try to come up with a clean one. Here's a joke: 

Why was the computer and the keyboard and the mouse all invited to the wedding? 

Because they were all already married. 

achieved tok/s: 19.62. Tokens: 54, seconds: 2.75

@stratika stratika requested a review from mikepapadim June 5, 2025 12:35
@stratika stratika self-assigned this Jun 5, 2025
@stratika stratika added the enhancement New feature or request label Jun 5, 2025
Copy link
Member

@mikepapadim mikepapadim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@mikepapadim mikepapadim merged commit 994d5c3 into beehive-lab:main Jun 5, 2025
1 check passed
@stratika stratika deleted the feat/windows-support branch June 30, 2025 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request windows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants