Skip to content

Allow env variable to override metal resource path #1415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 3, 2023

Conversation

codesoda
Copy link
Contributor

@codesoda codesoda commented Nov 2, 2023

Implementation for #1397

If an ENV variable called WHISPER_METAL_PATH_RESOURCES is available, then the ggml-metal.metal file is loaded from that path instead of using the bundle#pathForResource method.


Testing

🟥 Run main and expect a load (null) error.

Use make to compile the main binary and move it to a folder outside the project root.

$ ./main -m models/ggml-base.bin test.wav
...
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M1
ggml_metal_init: picking default device: Apple M1
ggml_metal_init: loading '(null)'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=258 "The file name is invalid."
whisper_init_state: ggml_metal_init() failed
error: failed to initialize whisper context

🟩 Launch again, passing the project root path, where the ggml-metal.metal file exists. Success!

$ WHISPER_METAL_PATH_RESOURCES=/Users/codesoda/projects/whisper.cpp ./main  -m models/ggml-base.bin test.wav
...
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M1
ggml_metal_init: picking default device: Apple M1
ggml_metal_init: loading '/Users/codesoda/projects/whisper.cpp/ggml-metal.metal'
ggml_metal_init: loaded kernel_add                            0x155707730 | th_max = 1024 | th_width =   32
...

@codesoda codesoda changed the title Allow env variable to override resource path Allow env variable to override metal resource path Nov 2, 2023
@ggerganov
Copy link
Member

Thanks for fixing this. I'm bringing a large update in #1422
Would you mind rebasing and adjusting the PR to that branch to reduce conflicts?

@codesoda codesoda force-pushed the metal_resource_path branch from 5c89856 to c057738 Compare November 3, 2023 10:42
@codesoda codesoda changed the base branch from master to sync November 3, 2023 10:43
@codesoda
Copy link
Contributor Author

codesoda commented Nov 3, 2023

Hope that update helps @ggerganov

@ggerganov ggerganov merged commit 592aed1 into ggml-org:sync Nov 3, 2023
@codesoda codesoda deleted the metal_resource_path branch November 3, 2023 13:24
ggerganov added a commit that referenced this pull request Nov 3, 2023
* sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.)

* metal : allow env metal variable to override resource path (#1415)

* Allow env variable to override resource path

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <[email protected]>

* sync : restore common / main from `master`

* sync : restore whisper from `master`

* talk-llama : update to latest llama.cpp

* ruby : fix build

* ggml : fix 32-bit ARM build

* ggml : fix MIN / MAX macro collisions + update ios bindings

* ggml : fix ifdefs and MIN / MAX again

* exampels : fix Obj-C and Swift examples

* ggml : fix 32-bit ARM compatibility

* ggml : one more attempt to fix 32-bit ARM compat

* whisper : fix support for larger graphs

---------

Co-authored-by: Chris Raethke <[email protected]>
vonstring pushed a commit to vonstring/whisper.cpp that referenced this pull request Nov 7, 2023
…-org#1422)

* sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.)

* metal : allow env metal variable to override resource path (ggml-org#1415)

* Allow env variable to override resource path

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <[email protected]>

* sync : restore common / main from `master`

* sync : restore whisper from `master`

* talk-llama : update to latest llama.cpp

* ruby : fix build

* ggml : fix 32-bit ARM build

* ggml : fix MIN / MAX macro collisions + update ios bindings

* ggml : fix ifdefs and MIN / MAX again

* exampels : fix Obj-C and Swift examples

* ggml : fix 32-bit ARM compatibility

* ggml : one more attempt to fix 32-bit ARM compat

* whisper : fix support for larger graphs

---------

Co-authored-by: Chris Raethke <[email protected]>
felrock pushed a commit to felrock/whisper.cpp that referenced this pull request Nov 18, 2023
…-org#1422)

* sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.)

* metal : allow env metal variable to override resource path (ggml-org#1415)

* Allow env variable to override resource path

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <[email protected]>

* sync : restore common / main from `master`

* sync : restore whisper from `master`

* talk-llama : update to latest llama.cpp

* ruby : fix build

* ggml : fix 32-bit ARM build

* ggml : fix MIN / MAX macro collisions + update ios bindings

* ggml : fix ifdefs and MIN / MAX again

* exampels : fix Obj-C and Swift examples

* ggml : fix 32-bit ARM compatibility

* ggml : one more attempt to fix 32-bit ARM compat

* whisper : fix support for larger graphs

---------

Co-authored-by: Chris Raethke <[email protected]>
landtanin pushed a commit to landtanin/whisper.cpp that referenced this pull request Dec 16, 2023
…-org#1422)

* sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.)

* metal : allow env metal variable to override resource path (ggml-org#1415)

* Allow env variable to override resource path

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <[email protected]>

* sync : restore common / main from `master`

* sync : restore whisper from `master`

* talk-llama : update to latest llama.cpp

* ruby : fix build

* ggml : fix 32-bit ARM build

* ggml : fix MIN / MAX macro collisions + update ios bindings

* ggml : fix ifdefs and MIN / MAX again

* exampels : fix Obj-C and Swift examples

* ggml : fix 32-bit ARM compatibility

* ggml : one more attempt to fix 32-bit ARM compat

* whisper : fix support for larger graphs

---------

Co-authored-by: Chris Raethke <[email protected]>
iThalay pushed a commit to iThalay/whisper.cpp that referenced this pull request Sep 23, 2024
…-org#1422)

* sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.)

* metal : allow env metal variable to override resource path (ggml-org#1415)

* Allow env variable to override resource path

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <[email protected]>

* sync : restore common / main from `master`

* sync : restore whisper from `master`

* talk-llama : update to latest llama.cpp

* ruby : fix build

* ggml : fix 32-bit ARM build

* ggml : fix MIN / MAX macro collisions + update ios bindings

* ggml : fix ifdefs and MIN / MAX again

* exampels : fix Obj-C and Swift examples

* ggml : fix 32-bit ARM compatibility

* ggml : one more attempt to fix 32-bit ARM compat

* whisper : fix support for larger graphs

---------

Co-authored-by: Chris Raethke <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants