Error when trying to install package from github repo with submodule

[0xf0f]

Environment

• pip version: 19.1.1
• Python version: 3.7
• OS: Windows 10 x64

Description
Pip is unable to install the package at https://github.com/pyminimp3/pyminimp3.
There's a permission error while cloning its submodule. Judging by Host key verification failed, it looks like git tries to use SSH to clone it when the initial command is requesting https.

Expected behavior
Package is cloned and installed.

How to Reproduce
Run the command pip install git+https://github.com/pyminimp3/pyminimp3

Output

C:\Users\admin>pip install git+https://github.com/pyminimp3/pyminimp3
Collecting git+https://github.com/pyminimp3/pyminimp3
  Cloning https://github.com/pyminimp3/pyminimp3 to c:\users\admin\appdata\local\temp\pip-req-build-dw1g94wp
  Running command git clone -q https://github.com/pyminimp3/pyminimp3 'C:\Users\admin\AppData\Local\Temp\pip-req-build-dw1g94wp'
  Running command git submodule update --init --recursive -q
  Host key verification failed.
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  fatal: clone of 'git@github.com:lieff/minimp3.git' into submodule path 'C:/Users/admin/AppData/Local/Temp/pip-req-build-dw1g94wp/minimp3' failed
  Failed to clone 'minimp3'. Retry scheduled
  Host key verification failed.
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  fatal: clone of 'git@github.com:lieff/minimp3.git' into submodule path 'C:/Users/admin/AppData/Local/Temp/pip-req-build-dw1g94wp/minimp3' failed
  Failed to clone 'minimp3' a second time, aborting
ERROR: Command "git submodule update --init --recursive -q" failed with error code 1 in C:\Users\admin\AppData\Local\Temp\pip-req-build-dw1g94wp

[xavfernandez]

Hello, like the error says, pip simply calls git submodule update --init --recursive -q then git uses the submodule url provided by https://github.com/pyminimp3/pyminimp3/blob/7d0b3695a2ff768f7bfcb881b5e493faf76a1cfd/.gitmodules#L3

I don't think pip can avoid that ?

[cjerdonek]

The reporter also reported it in the package's tracker: pyminimp3/pyminimp3#1

[pganssle]

What is the recommended solution here? Change the submodule to use the HTTPS link?

[cjerdonek]

That would be the easiest way to solve the reporter's issue I'm sure. GitHub's docs suggest here that https urls are more widely supported:

Cloning with HTTPS URLs (recommended)
The https:// clone URLs are available on all repositories, public and private. These URLs work everywhere--even if you are behind a firewall or proxy. In certain cases, if you'd rather use SSH, you might be able to use SSH over the HTTPS port.

I do wonder now though if there's anything pip can do to make this work. For example, I see this open issue ("Pip - doesn't pass proxy settings through to git/hg etc."): #227

@0xf0f If you try to clone the ssh url coming from the submodule on the command-line manually yourself, what happens for you? Does it work -- are you prompted for anything?

You want to try this:

$ git clone git@github.com:lieff/minimp3.git

[0xf0f]

Cloning the submodule manually works with that command. Only thing I'm prompted for is my passphrase.
One thing I noticed is if you follow the link to the submodule from the repo's master branch you arrive at this page: https://github.com/lieff/minimp3/tree/66bf4ad047b471b6e3288f8c76a20737c078b9a3 which is an old version of the master branch. I'm not experienced enough to know why this is the case. Perhaps the submodule needs to be removed and readded with a different command?

[pradyunsg]

Only thing I'm prompted for is my passphrase.

This might be why the command fails -- pip does not forward stdin to the VCS backends (for a bunch of reasons including that it is non-deterministic and automation usually hangs if waiting on stdin input). That means prompt for the passphrase would not work.

---

Switching to https is a good workaround for this.

[cjerdonek]

Out of curiosity, now that you've typed your password, if you try again from scratch using pip, does it work?

which is an old version of the master branch.

This is because the parent repo is pointing to particular older commit. That's up to whoever maintains the parent repo.

@pradyunsg It might be possible to add support for interactive mode if some kind of --interactive option is present.