Installation on Windows failed because Visual Studio is not installed

[oobabooga]

Trying to install with

pip install llama-cpp-python==0.1.23

on Windows in a micromamba environment resulted in the following error. It seems like the package is looking for Visual Studio, which is not installed on my system.

Is it possible to make it such that the package can be installed without the need for Visual Studio?

(C:\Users\me\Downloads\oobabooga-windows\oobabooga-windows\installer_files\env) C:\Users\me\Downloads\oobabooga-windows\oobabooga-windows\text-generation-webui>pip install llama-cpp-python==0.1.23
Collecting llama-cpp-python==0.1.23
  Downloading llama_cpp_python-0.1.23.tar.gz (530 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 530.0/530.0 kB 504.2 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in c:\users\me\downloads\oobabooga-windows\oobabooga-windows\installer_files\env\lib\site-packages (from llama-cpp-python==0.1.23) (4.5.0)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [308 lines of output]


      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.

        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 17 2022 x64 v143' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator

          Visual Studio 17 2022

        could not find any instance of Visual Studio.



      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 17 2022 x64 v143' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.

        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator

          Visual Studio 16 2019

        could not find any instance of Visual Studio.



      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Ninja (Visual Studio 15 2017 x64 v141)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is unknown
      CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
        No CMAKE_C_COMPILER could be found.

        Tell CMake where to find the compiler by setting either the environment
        variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
        the compiler, or to the compiler name if it is in the PATH.


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja (Visual Studio 15 2017 x64 v141)' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 15 2017 x64 v141' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator

          Visual Studio 15 2017

        could not find any instance of Visual Studio.



      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 15 2017 x64 v141' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 17 2022 x64 v143)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Running

         'nmake' '-?'

        failed with:

         The system cannot find the file specified


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 17 2022 x64 v143)' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 16 2019 x64 v142)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Running

         'nmake' '-?'

        failed with:

         The system cannot find the file specified


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 16 2019 x64 v142)' generator - failure
      --------------------------------------------------------------------------------



      --------------------------------------------------------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 15 2017 x64 v141)' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Running

         'nmake' '-?'

        failed with:

         The system cannot find the file specified


      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'NMake Makefiles (Visual Studio 15 2017 x64 v141)' generator - failure
      --------------------------------------------------------------------------------

      ********************************************************************************
      scikit-build could not get a working generator for your system. Aborting build.

      Building windows wheels for Python 3.10 requires Microsoft Visual Studio 2022.
      Get it with "Visual Studio 2017":

        https://visualstudio.microsoft.com/vs/

      Or with "Visual Studio 2019":

          https://visualstudio.microsoft.com/vs/

      Or with "Visual Studio 2022":

          https://visualstudio.microsoft.com/vs/

      ********************************************************************************
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[MillionthOdin16]

Hmmm. I just went through this process. I'll try to see if I find anything

[MillionthOdin16]

Ohhhh, it looks like when they are installing the llama_cpp package we are also getting the llama.cpp repo in order to build the shared lib. But not everyone has their build environment configured, so it fails. It spams a bunch about visual studio, but the real issue is here:

-- The C compiler identification is unknown
CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
  No CMAKE_C_COMPILER could be found.

@abetlen do you agree?

Can you include prebuilt?

[abetlen]

Like pre-built binaries?

The issue is that the binaries will likely not be built with the correct optimizations for the users particular CPU which will likely result in much worse performance than the user expects.

What's the process for setting up the build environment on windows, could we add this to the docs?

[jllllll]

The compilers conda package includes a c compiler, but I'm unsure if it is enough to compile on Windows. When I tried it, it detected my VS Build Tools installation and used that instead. Can someone without Visual Studio try it? I don't have the time right now to set up a VM.

[MillionthOdin16]

I saw this, which might explain why it's going for visual studio. I don't know an easy way to get people setup with a way to build it though.

Yes, there is a Python package called setuptools that includes an extension called setuptools.msvc, which can help locate and configure the Microsoft Visual C++ Build Tools automatically when building extension modules on Windows.
However, it's important to note that setuptools.msvc can only help configure the build environment if the Microsoft Visual C++ Build Tools are already installed on the system. It does not install the build tools for the user. Therefore, users still need to install the build tools themselves as described in the previous response.

I see a lot of stuff about using 'wheels' for different systems. But I know basically nothing about that haha. I know lots of people use the prebuilt binaries from releases on llama.cpp, but getting them setup on windows for building isn't quick. Sometimes people aren't able to run the binaries (which in our case would be the lib). 🤔

[MillionthOdin16]

If we don't figure out a better solution maybe we can include a base lib that's compatible with many systems and either fall back to that if not able to build, or just use it and tell the user that they might be able to get better performance by compiling it on their machine.

[abetlen]

I think I'm just going to resolve this by adding a link to install instructions for MSVC C++ or the conda compilers package. I worry that with the pre-built wheels there'll either be bugs to worry about or the issues will get flooded with performance complaints.

@oobabooga can you try running micromamba install compilers -c conda-forge and then installing the llama-cpp-python package?

[MillionthOdin16]

okay, true hahaha

[abetlen]

Actually on second thought we could do wheels and just upload them to the Github releases section if that's what you meant. That way the default pip install llama-cpp-python can remain a source-only package and if anyone wants a wheel install for whatever reason they can pip install https://github.com/abetlen/llama-cpp-python/releases/download/release-name/llama-cpp-python.whl.

[MillionthOdin16]

I didn't really have a solution in mind haha. I just know from the gpt4all discord that lots of users aren't devs, so they like downloads. That seems like a good plan 👍

[oobabooga]

@jllllll @abetlen same error, but this time it only complained about VS 2019 and 2022 instead of every version since 1998

(C:\Users\me\Downloads\oobabooga-windows\oobabooga-windows\installer_files\env) C:\Users\me\Downloads\oobabooga-windows\oobabooga-windows>.\installer_files\mamba\micromamba.exe install compilers -c conda-forge

                                           __
          __  ______ ___  ____ _____ ___  / /_  ____ _
         / / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
        / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
       / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
      /_/

conda-forge/win-64                                          Using cache
conda-forge/noarch                                          Using cache

Pinned packages:
  - python 3.10.*


Transaction

  Prefix: C:\Users\me\Downloads\oobabooga-windows\oobabooga-windows\installer_files\env

  All requested packages already installed


Transaction starting
Transaction finished

(C:\Users\me\Downloads\oobabooga-windows\oobabooga-windows\installer_files\env) C:\Users\me\Downloads\oobabooga-windows\oobabooga-windows>pip install llama-cpp-python==0.1.23
Collecting llama-cpp-python==0.1.23
  Using cached llama_cpp_python-0.1.23.tar.gz (530 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in c:\users\me\downloads\oobabooga-windows\oobabooga-windows\installer_files\env\lib\site-packages (from llama-cpp-python==0.1.23) (4.5.0)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [49 lines of output]


      --------------------------------------------------------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      CMake Error at CMakeLists.txt:2 (PROJECT):
        Generator

          Visual Studio 16 2019

        could not find any instance of Visual Studio.



      -- Configuring incomplete, errors occurred!
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Visual Studio 16 2019 x64 v142' generator - failure
      --------------------------------------------------------------------------------

      ********************************************************************************
      scikit-build could not get a working generator for your system. Aborting build.

      Building windows wheels for Python 3.10 requires Microsoft Visual Studio 2022.
      Get it with "Visual Studio 2017":

        https://visualstudio.microsoft.com/vs/

      Or with "Visual Studio 2019":

          https://visualstudio.microsoft.com/vs/

      Or with "Visual Studio 2022":

          https://visualstudio.microsoft.com/vs/

      ********************************************************************************
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[jllllll]

@oobabooga Yeah, we'll either have to wait for abetlen to upload wheels that the install script can download, or I can make a wheel for us to use in the mean time. As long as the wheel is installed before installing requirements.txt, then you can add llama-cpp-python back into it.

[abetlen]

@jllllll working on publishing the wheels through Github releases, you should then be able to add the wheel urls to a requirements file.

If a user then wants a more optimized version and they have a c compiler installed they can just pip install --upgrade --no-deps --force-reinstall llama-cpp-python. Does that work?

[abetlen]

@jllllll @oobabooga when one of you gets the chace can you try to install a wheel from the artifacts built here https://github.com/abetlen/llama-cpp-python/actions/runs/4636312659

Sorry there's like a million of them, probably don't need to build for each version of python independently. I'll try to clean this up tomorrow. Cheers