Skip to content

Conversation

@iandoe
Copy link
Contributor

@iandoe iandoe commented Mar 21, 2024

What does this PR do?

This PR adds a Dockerfile-arm64 that does build on M1 Macs (and potentially on M2 as well, not tested)

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@iandoe iandoe marked this pull request as ready for review March 21, 2024 05:53
@iandoe iandoe changed the title Add Dockerfile-arm64 to allow docker builds on Mac M1/M2 series Add Dockerfile-arm64 to allow docker builds on Apple M1/M2 architecture Mar 21, 2024
@OlivierDehaene
Copy link
Contributor

While it does built, it's impossible to have Metal/accelerate. I don't really see why you would use such images. Do you have a use case in mind?

@iandoe
Copy link
Contributor Author

iandoe commented Mar 21, 2024

Hey @OlivierDehaene,

Thanks for having a look, TEI is great.

I'm using a local docker instance in development, while it is indeed pretty slow, it saves me from using my HF Inference Endpoints in development and incurring $$ costs, and facilitates setup with other engineers with docker-compose.

Self compiling with rust as explained in the local install does indeed allow for Metal support and is much faster. Speed is not of the essence in my use case (more so ease-of-setup).

Hope that helps and if you'd prefer not to merge i'm happy to close and just answer in #46.

Regards.

@OlivierDehaene
Copy link
Contributor

Ok that makes sense. If you can add a disclaimer in the README explaining the issue with Metal/accelerate in ARM builds that would be great!
We can even build the image in the CI I guess.

Copy link
Contributor

@OlivierDehaene OlivierDehaene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@OlivierDehaene OlivierDehaene merged commit 6fa3c6a into huggingface:main Mar 21, 2024
MasakiMu319 pushed a commit to MasakiMu319/text-embeddings-inference that referenced this pull request Nov 27, 2024
aagnone3 pushed a commit to StratisLLC/hf-text-embeddings-inference that referenced this pull request Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants