Skip to content

Docker/rootless: "ssh-keygen": executable file not found in $PATH - #23363

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
williamdes opened this issue Mar 7, 2023 · 16 comments · Fixed by #23798
Closed

Docker/rootless: "ssh-keygen": executable file not found in $PATH - #23363

williamdes opened this issue Mar 7, 2023 · 16 comments · Fixed by #23798
Labels

Comments

@williamdes
Copy link

williamdes commented Mar 7, 2023

Description

Cannot verify your SSH key: SSHKeyGenParsePublicKey: fail to parse public key: exec: "ssh-keygen": executable file not found in $PATH -

After clicking on "Add deploy key" with the form in the screenshot

On https://git.domain.tld/<org>/<repo>/settings/keys

That said I tried to verify my SSH key that I had added to my profile and it worked fine, so maybe it uses another code function.

Gitea Version

1.19.0-rc1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

image

Git Version

No response

Operating System

No response

How are you running Gitea?

With Docker/docker-compose

version: "2.3"

services:
    my_git:
        image: gitea/gitea:1.19.0-rc1-rootless

I am pretty sure it works fine with gitea/gitea:1.19.0-rc1

Database

MySQL

@williamdes
Copy link
Author

williamdes commented Mar 7, 2023

Quick fix: docker exec -u root -it git.domain.tld apk add openssh-keygen
No restart needed

Edit: maybe it also does not work with gitea/gitea:1.19.0-rc1, please try

@wxiaoguang
Copy link
Contributor

It looks like a bug in code if you didn't disable the builtin ssh server.

The code works like this:

  • If you disabled builtin ssh server, then use ssh command line to verify
  • If you enabled builtin ssh server, then use gitea internal ssh package to verify

If you have enabled the builtin ssh server (didn't change any default config), then it must be a regression bug.

@williamdes
Copy link
Author

It looks like a bug in code if you didn't disable the builtin ssh server.

I can confirm I did not disable it, I use the SSH server with docker pass though (#19065 (comment)): https://docs.gitea.io/en-us/install-with-docker/#docker-shell-with-authorized_keys

        environment:
            # https://github.com/go-gitea/gitea/blob/release/v1.19/docker/rootless/usr/local/bin/gitea#L40
            GITEA_WORK_DIR: /data/gitea
            # https://github.com/go-gitea/gitea/blob/release/v1.19/docker/rootless/usr/local/bin/gitea#L35
            GITEA_APP_INI: /etc/gitea/app.ini
            GITEA__cron__ENABLED: "true"
            GITEA__server__SSH_DOMAIN: git-ssh.domain.tld
            GITEA__server__SSH_USER: git
            GITEA__server__SSH_PORT: 22
            GITEA__server__ROOT_URL: https://git.domain.tld/
            GITEA__server__HTTP_PORT: ${GITEA_HTTP_PORT:-3000}
            GITEA__server__DISABLE_SSH: ${GITEA_DISABLE_SSH:-true}
            GITEA__server__SSH_LISTEN_HOST: ${GITEA_SSH_LISTEN_HOST:-0.0.0.0}
            GITEA__server__SSH_LISTEN_PORT: ${GITEA_SSH_LISTEN_PORT:-2222}
            GITEA__server__APP_DATA_PATH: /data/gitea
            GITEA__server__SSH_ROOT_PATH: /home/git/.ssh
            GITEA__log__LEVEL: "Debug"
            GITEA__log__ROOT_PATH: /data/git/log
            GITEA__lfs__PATH: /data/git/lfs
            GITEA__repository__ROOT: /data/git/repositories
            GITEA__database__DB_TYPE: mysql
            GITEA__database__HOST: "${GITEA_DB_HOST}"
            GITEA__database__NAME: "${GITEA_DB_NAME}"
            GITEA__database__USER: "${GITEA_DB_USER}"
            GITEA__database__PASSWD: "${GITEA_DB_PASS}"
            GITEA__session__COOKIE_SECURE: "true"
            GITEA__session__SAME_SITE: strict
            GITEA__mailer__ENABLED: "true"
            GITEA__mailer__FROM: ${GITEA_MAIL_FROM}
            GITEA__mailer__GITEA_MAIL_SUBJECT_PREFIX: ${GITEA_MAIL_SUBJECT_PREFIX}
            GITEA__mailer__HELO_HOSTNAME: ${GITEA_MAIL_HELO_HOSTNAME}
            GITEA__mailer__PROTOCOL: smtps
            GITEA__mailer__SMTP_ADDR: ${GITEA_MAIL_SMTP_ADDR}
            GITEA__mailer__SMTP_PORT: 465
            GITEA__mailer__USER: ${GITEA_MAIL_USER}
            GITEA__mailer__PASSWD: "${GITEA_MAIL_PASSWORD}"
            GITEA__service__DISABLE_REGISTRATION: ${GITEA_DISABLE_REGISTRATION:-true}
            GITEA__service__NO_REPLY_ADDRESS: ${GITEA_NO_REPLY_ADDRESS}
            GITEA__actions__ENABLED: "true"
            GITEA__federation__ENABLED: "true"
            GITEA__cron.update_checker__ENABLED: "true"

@wxiaoguang
Copy link
Contributor

wxiaoguang commented Mar 7, 2023

What's the content of your app.ini?

Your config GITEA__server__DISABLE_SSH: ${GITEA_DISABLE_SSH:-true} doesn't look right.

By default, the app.ini looks like this, check the DISABLE_SSH = false and START_SSH_SERVER = true

[server]
APP_DATA_PATH           = /var/lib/gitea
SSH_DOMAIN              = localhost
HTTP_PORT               = 3000
ROOT_URL                = http://localhost:3000/
DISABLE_SSH             = false
; In rootless gitea container only internal ssh server is supported
START_SSH_SERVER        = true
SSH_PORT                = 2222
SSH_LISTEN_PORT         = 2222
BUILTIN_SSH_SERVER_USER = git
LFS_START_SERVER        = true
DOMAIN                  = localhost

@williamdes
Copy link
Author

Your config GITEA__server__DISABLE_SSH: ${GITEA_DISABLE_SSH:-true} doesn't look right.

It just says to use the ENV value of GITEA_DISABLE_SSH or "true". It's false in my ENV.

GITEA_DISABLE_SSH=false
GITEA_DISABLE_REGISTRATION=true
APP_NAME = <redacted>
RUN_MODE = prod
RUN_USER = git

[repository]
ROOT = /data/git/repositories

[repository.local]
LOCAL_COPY_PATH = /data/gitea/tmp/local-repo

[repository.upload]
TEMP_PATH = /data/gitea/uploads

[server]
APP_DATA_PATH    = /data/gitea
DOMAIN           = localhost
SSH_DOMAIN       = <redacted>
HTTP_PORT        = 3000
ROOT_URL         = https://<redacted>
DISABLE_SSH      = false
SSH_PORT         = 22
SSH_LISTEN_PORT  = 2222
LFS_START_SERVER = true
LFS_JWT_SECRET   = <redacted>
OFFLINE_MODE     = true
SSH_USER         = git
SSH_LISTEN_HOST  = 0.0.0.0
SSH_ROOT_PATH    = /home/git/.ssh

[database]
PATH     = /data/gitea/gitea.db
DB_TYPE  = mysql
HOST     = <redacted>
NAME     = <redacted>
USER     = <redacted>
PASSWD   = <redacted>
LOG_SQL  = false
SCHEMA   = 
SSL_MODE = disable
CHARSET  = utf8mb4

[indexer]
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve

[session]
PROVIDER_CONFIG = /data/gitea/sessions
PROVIDER        = file
SAME_SITE       = strict
COOKIE_SECURE   = true

[picture]
AVATAR_UPLOAD_PATH            = /data/gitea/avatars
REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars
DISABLE_GRAVATAR              = true
ENABLE_FEDERATED_AVATAR       = false

[attachment]
PATH = /data/gitea/attachments

[log]
MODE      = console
LEVEL     = Debug
ROUTER    = console
ROOT_PATH = /data/git/log

[security]
INSTALL_LOCK                  = true
SECRET_KEY                    = <redacted>
REVERSE_PROXY_LIMIT           = 1
REVERSE_PROXY_TRUSTED_PROXIES = *
INTERNAL_TOKEN                = <redacted>
PASSWORD_HASH_ALGO            = pbkdf2

[service]
DISABLE_REGISTRATION              = true
REQUIRE_SIGNIN_VIEW               = true
REGISTER_EMAIL_CONFIRM            = true
ENABLE_NOTIFY_MAIL                = false
ALLOW_ONLY_EXTERNAL_REGISTRATION  = false
ENABLE_CAPTCHA                    = false
DEFAULT_KEEP_EMAIL_PRIVATE        = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
DEFAULT_ENABLE_TIMETRACKING       = false
NO_REPLY_ADDRESS                  = <redacted>

[oauth2]
JWT_SECRET = <redacted>

[mailer]
ENABLED                   = true
FROM                      = <redacted>
USER                      = <redacted>
PASSWD                    = <redacted>
HELO_HOSTNAME             = <redacted>
PROTOCOL                  = smtps
GITEA_MAIL_SUBJECT_PREFIX = <redacted>
SMTP_PORT                 = 465
SMTP_ADDR                 = <redacted>

[openid]
ENABLE_OPENID_SIGNIN = false
ENABLE_OPENID_SIGNUP = false

[cron]
ENABLED = true

[federation]
ENABLED = true

[actions]
ENABLED = true

[cron.update_checker]

[lfs]
PATH = /data/git/lfs

@wxiaoguang
Copy link
Contributor

You need START_SSH_SERVER=true but I didn't see it in your config.

@williamdes
Copy link
Author

williamdes commented Mar 8, 2023

You need START_SSH_SERVER=true but I didn't see it in your config.

The server is started because I use SSH git clone and it works fine 🤔
And when gitea starts it says it listens on port 22

Edit: I need to double check that, but is is really important ?
My demonstration of a quick fix shows the Docker image has a missing package, that's it ?
Right ?

@wxiaoguang
Copy link
Contributor

Nope, by design, the docker-rootless shouldn't have that package. Instead, Gitea rootless should use internal ssh package to verify.

@williamdes
Copy link
Author

williamdes commented Mar 8, 2023

Nope, by design, the docker-rootless shouldn't have that package. Instead, Gitea rootless should use internal ssh package to verify.

Oh okay, so I guess the code should be adapted anyway ?

Edit: can you explain why ?

@wxiaoguang
Copy link
Contributor

wxiaoguang commented Mar 8, 2023

More information:

Gitea Docker and Gitea Docker-rootless are totally different for SSH problem.

  • Gitea Docker (aka gitea/gitea:1.19.0-rc1), it uses OpenSSH server.
  • Gitea Docker (aka gitea/gitea:1.19.0-rc1-rootless), it uses builtin SSH server (no OpenSSH).

Oh okay, so I guess the code should be adapted anyway ?

Yup, that why you must have START_SSH_SERVER=true in config (if you are using rootless), to tell Gitea to use internal ssh package.


Update 2: I agree that this problem is quite annoying and unfriendly to end-users ..... I do not like the inconsistent behaviors either, but the situation has been like this .... unless some one would have time to improve it.

@wxiaoguang
Copy link
Contributor

And there is still a question I haven't understood. By default, when you start the rootless version, there is a START_SSH_SERVER=true in the config (according to my local test), but I didn't see it in your config. Do you know what causes this problem? Or did you re-use the config from a non-rootless setup?

@williamdes
Copy link
Author

And there is still a question I haven't understood. By default, when you start the rootless version, there is a START_SSH_SERVER=true in the config (according to my local test), but I didn't see it in your config. Do you know what causes this problem? Or did you re-use the config from a non-rootless setup?

Very good question, in fact my setup was root and I just switched it to rootless when trying to debug stuff. So maybe this creates a side effect somewhere

@yogo1212
Copy link
Contributor

same issue here.
hope that fixes it.

@wxiaoguang
Copy link
Contributor

same issue here. hope that fixes it.

set START_SSH_SERVER=true, does it help?

@yogo1212
Copy link
Contributor

hi 👋

likely, it will. but that's off the point.
i don't want an ssh server inside the container :-)

@yogo1212
Copy link
Contributor

i'm currently running this: #23798

yogo1212 added a commit to yogo1212/gitea that referenced this issue Apr 10, 2023
Without this patch, the setting SSH.StartBuiltinServer decides whether the
native (Go) implementation is used rather than calling 'ssh-keygen'.
It's possible for 'using ssh-keygen' and 'using the built-in server' to be
independent.
In fact, the gitea rootless container doesn't ship ssh-keygen and can be
configured to use the host's SSH server - which will cause the public key
parsing mechanism to break.

This commit changes the decision to be based on SSH.KeygenPath instead.
Any existing configurations with a custom KeygenPath set will continue to
function.  The new default value of '' selects the native version.
The downside of this approach is that anyone who has relying on plain
'ssh-keygen' to have special properties will now be using the native version
instead.
I assume the exec-variant is only there because /x/crypto/ssh didn't support
ssh-ed25519 until 2016. I don't see any other reason for using it.

Fixes go-gitea#23363

Co-authored-by: wxiaoguang <[email protected]>
Signed-off-by: Leon M. Busch-George <[email protected]>
lunny pushed a commit that referenced this issue Apr 11, 2023
Without this patch, the setting SSH.StartBuiltinServer decides whether
the native (Go) implementation is used rather than calling 'ssh-keygen'.
It's possible for 'using ssh-keygen' and 'using the built-in server' to
be independent.
In fact, the gitea rootless container doesn't ship ssh-keygen and can be
configured to use the host's SSH server - which will cause the public
key parsing mechanism to break.

This commit changes the decision to be based on SSH.KeygenPath instead.
Any existing configurations with a custom KeygenPath set will continue
to function. The new default value of '' selects the native version. The
downside of this approach is that anyone who has relying on plain
'ssh-keygen' to have special properties will now be using the native
version instead.
I assume the exec-variant is only there because /x/crypto/ssh didn't
support ssh-ed25519 until 2016. I don't see any other reason for using
it so it might be an acceptable risk.

Fixes #23363

EDIT: this message was garbled when I tried to get the commit
description back in.. Trying to reconstruct it:

## ⚠️ BREAKING ⚠️ Users who don't have SSH.KeygenPath
explicitly set and rely on the ssh-keygen binary need to set
SSH.KeygenPath to 'ssh-keygen' in order to be able to continue using it
for public key parsing.

There was something else but I can't remember at the moment.

EDIT2: It was about `make test` and `make lint`. Can't get them to run.
To reproduce the issue, I installed `golang` in `docker.io/node:16` and
got:
```
...
go: mvdan.cc/xurls/[email protected]: unknown revision mvdan.cc/xurls/v2.4.0
go: gotest.tools/[email protected]: unknown revision gotest.tools/v3.4.0
...
go: gotest.tools/[email protected]: unknown revision gotest.tools/v3.0.3
...
go: error loading module requirements
```

Signed-off-by: Leon M. Busch-George <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants