[2.23.0] bash: /dev/fd/62: No such file or directory

[JoseGalRe]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

git version 2.23.0.windows.1
cpu: x86_64
built from commit: 4db2e5cc9e1522131a039cbad3970f147a39f0ce
sizeof-long: 4
sizeof-size_t: 8

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

Microsoft Windows [Version 10.0.17763.678]

• What options did you set as part of the installation? Or did you choose the
  defaults?

Editor Option: Notepad++
Custom Editor Path:
Path Option: Cmd
SSH Option: OpenSSH
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Enabled
Enable Builtin Interactive Add: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Nothing

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

git-bash.exe

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

cat < <(echo "hello")

• What did you expect to occur after running these commands?

hello

• What actually happened instead?

bash: /dev/fd/62: No such file or directory

Note: on '2.22.0' works fine

[PhilipOakley]

In a dumb, just copy the commands, mode, I was able to repeat the exercise.

I then tried:

$ <(echo "hello")
bash: /dev/fd/63: No such file or directory

(the fd number is one higher, 63 rather than 62.)
ASCII 62 is 3E ">" if that has any relevance.

I suspect that this because of the Cygwin update in the release notes.

However, as someone who is not a happy scripter, I wasn't sure what the command hoped to do anyway. Perhaps you could describe your expectation and decompose what you hoped it would do?

[dscho]

Can you open Git Bash as administrator, call mkdir -p /dev/fd, and then try again (not as admin)?

[PhilipOakley]

@dscho On my machine (git opened as admin):

phili@Philip-Win10 MINGW64 ~
$  mkdir -p /dev/fd
mkdir: cannot create directory ‘/dev/fd’: Read-only file system

phili@Philip-Win10 MINGW64 ~
$ git version --build-options
git version 2.23.0.windows.1
cpu: x86_64
built from commit: 4db2e5cc9e1522131a039cbad3970f147a39f0ce
sizeof-long: 4
sizeof-size_t: 8

phili@Philip-Win10 MINGW64 ~
$

Just a data point...

[JoseGalRe]

In a dumb, just copy the commands, mode, I was able to repeat the exercise.

I then tried:

$ <(echo "hello")
bash: /dev/fd/63: No such file or directory

(the fd number is one higher, 63 rather than 62.)
ASCII 62 is 3E ">" if that has any relevance.

I suspect that this because of the Cygwin update in the release notes.

However, as someone who is not a happy scripter, I wasn't sure what the command hoped to do anyway. Perhaps you could describe your expectation and decompose what you hoped it would do?

of course the command with "cat" is just an example, I use "< <(..)" usually to insert elements in an array, but "cat" is a simple and short example.

yes, this breaks all my scripts, where I use "< <(..)"
I have read the release notes and I also think it is due to the Cygwin update.

Can you open Git Bash as administrator, call mkdir -p /dev/fd, and then try again (not as admin)?

whether admin or not, it is in read-only mode

I tried:

mkdir -p /dev/fd
ln -s /proc/self/fd /dev/fd

but the answer is always the same:

read-only file system

[dscho]

Okay, this is what I did, inspired by https://sourceware.org/ml/cygwin/2009-02/msg00389.html:

1. as administrator, in a PowerShell, I called mkdir C:\Program Files\Git\dev.
2. as administrator, in a Git Bash, I called ln -s /proc/self/fd /dev/fd

After that, things seemed to work for me.

Note: it is important to create the directory outside of Git Bash, as the MSYS2 runtime pretends that that directory exists even if it doesn't. And the ln call fails unless the directory really exists, so it needs to be created explicitly.

Can you try that? If that fixes the problems, maybe you would be interested in working on a patch to install.iss to fix this for good?

[stepro]

This has broken many of my scripts as well. Seems there should be a bash postinstall script like this that should run to automatically pre-create these device links. What changed between 2.22 and 2.23?

[JoseGalRe]

Okay, this is what I did, inspired by https://sourceware.org/ml/cygwin/2009-02/msg00389.html:

1. as administrator, in a PowerShell, I called mkdir C:\Program Files\Git\dev.
2. as administrator, in a Git Bash, I called ln -s /proc/self/fd /dev/fd

After that, things seemed to work for me.

Note: it is important to create the directory outside of Git Bash, as the MSYS2 runtime pretends that that directory exists even if it doesn't. And the ln call fails unless the directory really exists, so it needs to be created explicitly.

Thanks mate, now works perfectly!
maybe in the next releases the installer should create the dev dir and make the symbolic link? xD

Can you try that? If that fixes the problems, maybe you would be interested in working on a patch to install.iss to fix this for good?

I don't know very well how git-bash could run as an administrator on inno setup during installation to run a postinstall as stepro says

[PhilipOakley]

1. Opened PowerShell as administrator, I called mkdir "C:\Program Files\Git\dev" (with quotes).
2. Opened a Git Bash as administrator, I called ln -s /proc/self/fd /dev/fd

Also worked for me [my data point]. I.e.

phili@Philip-Win10 MINGW64 /c/git-sdk-64/usr/src/git ((a85708cd88...))
$ cat < <(echo "hello")
hello

Earlier, in a regular (non-admin) bash shell (V2.23.0) I'd seen, (by chance, I'd forgotten to cd back to my repo)

phili@Philip-Win10 MINGW64 /c/git-sdk-64/usr/src/git ((a85708cd88...))
$ cd /dev/

phili@Philip-Win10 MINGW64 /dev
$ ls
clipboard  console  null  random  sda1  sda4  sdb2  ttyS0    zero
conin      dsp      ptmx  scd0    sda2  sdb   sr0   urandom
conout     full     pty0  sda     sda3  sdb1  tty   windows

phili@Philip-Win10 MINGW64 /dev
$ git help config
Error: Current working directory is a virtual Cygwin directory which does
not exist for a native Windows application.
Can't start native Windows application from here.

which at least confirms the underlying Cygwin aspect of the virtual file system at that point.

[stepro]

I don't know very well how git-bash could run as an administrator on inno setup during installation to run a postinstall as stepro says

Clearly some part of the 2.22 installation used to do this. I reverted back and my /dev directory looks like this:

drwxr-xr-x 1 stepro 1049089        0 Aug 19 16:16 ./
drwxr-xr-x 1 stepro 1049089        0 Aug 19 15:23 ../
crw-rw-rw- 1 stepro 1049089  13, 254 Nov 30  2006 clipboard
crw-rw-rw- 1 stepro 1049089   5, 255 Aug 20 08:56 conin
crw-rw-rw- 1 stepro 1049089   5, 254 Aug 20 08:56 conout
crw-rw-rw- 1 stepro 1049089   5,   1 Aug 20 08:56 console
crw-rw-rw- 1 stepro 1049089  14,   3 Aug 20 08:56 dsp
lrwxrwxrwx 1 stepro 1049089       13 Aug 19 14:08 fd -> /proc/self/fd/
crw-rw-rw- 1 stepro 1049089   1,   7 Aug 20 08:56 full
drwxr-xr-x 1 stepro 1049089        0 Aug 19 14:08 mqueue/
crw-rw-rw- 1 stepro 1049089   1,   3 Aug 20 08:56 null
crw-rw-rw- 1 stepro 1049089   5,   2 Aug 20 08:56 ptmx
crw------- 1 stepro 1049089 136,   0 Aug 20 08:56 pty0
crw-rw-rw- 1 stepro 1049089   1,   8 Aug 20 08:56 random
brw-rw-rw- 1 stepro 1049089   8,   0 Aug 20 08:56 sda
brw-rw-rw- 1 stepro 1049089   8,   1 Aug 20 08:56 sda1
brw-rw-rw- 1 stepro 1049089   8,   2 Aug 20 08:56 sda2
brw-rw-rw- 1 stepro 1049089   8,   3 Aug 20 08:56 sda3
brw-rw-rw- 1 stepro 1049089   8,   4 Aug 20 08:56 sda4
brw-rw-rw- 1 stepro 1049089   8,  16 Aug 20 08:56 sdb
brw-rw-rw- 1 stepro 1049089   8,  17 Aug 20 08:56 sdb1
brw-rw-rw- 1 stepro 1049089   8,  18 Aug 20 08:56 sdb2
drwxr-xr-x 1 stepro 1049089        0 Aug 19 14:08 shm/
lrwxrwxrwx 1 stepro 1049089       15 Aug 19 14:08 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 stepro 1049089       15 Aug 19 14:08 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 stepro 1049089       15 Aug 19 14:08 stdout -> /proc/self/fd/1
crw-rw-rw- 1 stepro 1049089   5,   0 Aug 20 08:56 tty
crw-rw-rw- 1 stepro 1049089   1,   9 Aug 20 08:56 urandom
crw-rw-rw- 1 stepro 1049089  13, 255 Aug 20 08:56 windows
crw-rw-rw- 1 stepro 1049089   1,   5 Aug 20 08:56 zero

In the new installation, there are no symbolic links.

Workarounds are great, but this needs to be addressed at the source. Fundamental aspects of Bash assume the existence of these links, as pointed out here, e.g.:

# Install /dev/fd, /dev/std{in,out,err}.  The bash builtin test was compiled
# to assume these exist, so use /bin/test to really check.

[dscho]

The solution will be to have the InnoSetup-based installer create this directory before the post-install script runs (I already pointed out why that shell script cannot create it).

I trust you folks to make this happen, in a PR to the build-extra repository, changing install.iss accordingly.

[stepro]

This doesn't fix the root cause, since install.iss never created this directory before - something else did. And whatever that is/was (MSYS2, Cygwin, something else? I don't understand all these dependencies) is also now broken. Shouldn't we understand the root of the problem before patching elsewhere?

[dakotahawkins]

The latest installs of cygwin and msys2 seem fine, /dev/fd -> /proc/self/fd.

[dscho]

Shouldn't we understand the root of the problem before patching elsewhere?

@stepro go to town, then! For starters, you might want to unpack portable versions of the working vs non-working Git for Windows, then investigate the differences in the /etc/package-versions.txt file, and based on that, replace files in the non-working versions until things work.

You will want to unpack the portable Git's via 7-Zip, though, and then prevent the /etc/post-install/99-post-install-cleanup.post script from removing the post-install scripts (by deleting it before running git-bash.exe). I see, for example, that the /dev/fd link itself is created in /etc/post-install/01-devices.post.

[stepro]

Thanks @dscho, these links are very helpful. I'll take a look.

[stepro]

Ok, the root cause comes from this change, which alters the behavior of mkdir such that it will no longer create an entry in the Windows filesystem for Cygwin-only directories like /dev. This commit was introduced in Git for Windows as part of the upgrade to by moving to msys2-runtime v3.0.7 patch 3.

Previously, mkdir succeeded, allowing this line to both create /dev and successfully add files like /dev/fd to it. Now, it fails to create the directory with the error mkdir: cannot create directory '/dev': File exists (although this error is not seen because it pipes stderr to /dev/null), and the script proceeds because /dev does effectively exist and passes the existence check. However, by not being an actual directory in the Windows filesystem, Cygwin doesn't allow creation of any files in that directory, and returns a "Read-only file system" error. Therefore creation of the shm and mqueue directories fail, and adding the various /dev/fd links also fails. When this is called by post-install.bat, the output is:

"running post-install"
mkdir: cannot create directory '/dev/shm': Read-only file system

Creating /dev/shm directory failed.
POSIX semaphores and POSIX shared memory will not work

mkdir: cannot create directory '/dev/mqueue': Read-only file system

Creating /dev/mqueue directory failed.
POSIX message queues will not work

ln: failed to create symbolic link '/dev/stdin': Read-only file system
ln: failed to create symbolic link '/dev/stdout': Read-only file system
ln: failed to create symbolic link '/dev/stderr': Read-only file system
ln: failed to create symbolic link '/dev/fd': Read-only file system
'C:\windows\system32\drivers\etc\hosts' -> '/etc/hosts'
'C:\windows\system32\drivers\etc\protocol' -> '/etc/protocols'
'C:\windows\system32\drivers\etc\services' -> '/etc/services'
'C:\windows\system32\drivers\etc\networks' -> '/etc/networks'

Because this script has no error handling, the Git for Windows installer thinks that post-install.bat succeeded when really it should have failed.

Why is this not failing in MSYS2 itself? Because the current installer is from May 24 and the patch wasn't committed to Cygwin until June 5.

Why is this not failing in Cygwin installs? Same deal - the latest Cygwin installer puts down a cygwin1.dll (where the compiled code that was patched lives) from April 30.

So, this problem is likely to start showing up in MSYS2 and Cygwin if they don't do something about it. Who can be contacted about ensuring those projects don't break?

Git for Windows seems ahead of the game by shipping a newer version of Cygwin than Cygwin itself that already manifests the issue. If folks from Cygwin agree that the right fix for this is for a given installer to manually create the directory beforehand, then that's what should be done. Otherwise, it could be added to the Git for Windows installer as a temporary workaround until a new version of Cygwin is picked up that properly accounts for it.