runtime: windows DLL mappings interfere with 32-bit memory allocation

[gopherbot]

by zippoxer:

Before filing a bug, please check whether it has been fixed since
the latest release: run "hg pull", "hg update default", rebuild, and
retry
what you did to
reproduce the problem.  Thanks.

What steps will reproduce the problem?
1. Installing gomingw installer for build of release r60.1
2. Executing make, then make install in the (attached to this issue) package's directory.
3. Testing the installed package by calling Now().

What is the expected output?
The x position of the cursor.

What do you see instead?
An error:
>> throw: runtime: cannot reserve arena virtual address space

Which compiler are you using (5g, 6g, 8g, gccgo)?
8g

Which operating system are you using?
Windows 7

Which revision are you using?  (hg identify)
I ran "Installer for Win32 (experimental) build of release.r60.1"
(hd identify doesn't cooperate, sorry)

Please provide any additional information below.

Attachments:

1. somecgopackage.go (188 bytes)
2. Makefile (122 bytes)

[alexbrainman]

Comment 1:

I can't reproduce your problem on Windows 7:
$ cat Makefile
include $(GOROOT)/src/Make.inc
TARG=a
CGOFILES=\
        a.go\
include $(GOROOT)/src/Make.pkg
$ cat a.go
package a
/*
#include <windows.h>
int xPlease() {
        POINT p;
        GetCursorPos(&p);
        return p.x;
}
*/
import "C"
func Now() {
        println("I got that. It's: ", int(C.xPlease()))
}
$ cat a_test.go
package a_test
import (
        "a"
        "testing"
)
func TestA(t *testing.T) {
        a.Now()
}
$ make test
gotest
rm -f _test/a.a
CGOPKGPATH= cgo --  a.go
touch _obj/_cgo_run
8g  -o _gotest_.8  _obj/a.cgo1.go _obj/_cgo_gotypes.go
8c -FVw -Ic:/MinGW/go/pkg/windows_386 -I . -o "_cgo_defun.8" _obj/_cgo_defun.c
gcc -m32 -I . -g -fPIC -O2 -o _cgo_main.o -c   _obj/_cgo_main.c
_obj/_cgo_main.c:1:0: warning: -fPIC ignored for target (all code is position
independent)
gcc -m32 -I . -g -fPIC -O2 -o a.cgo2.o -c   _obj/a.cgo2.c
_obj/a.cgo2.c:1:0: warning: -fPIC ignored for target (all code is position independent)
gcc -m32 -I . -g -fPIC -O2 -o _cgo_export.o -c   _obj/_cgo_export.c
_obj/_cgo_export.c:1:0: warning: -fPIC ignored for target (all code is position
independent)
gcc -m32 -g -fPIC -O2 -o _cgo1_.o _cgo_main.o a.cgo2.o _cgo_export.o
cgo -dynimport _cgo1_.o >_obj/_cgo_import.c_ && mv -f _obj/_cgo_import.c_
_obj/_cgo_import.c
8c -FVw -I . -o "_cgo_import.8" _obj/_cgo_import.c
rm -f _test/a.a
gopack grc _test/a.a _gotest_.8  _cgo_defun.8 _cgo_import.8 a.cgo2.o _cgo_export.o
I got that. It's:  1115
PASS
$

Owner changed to @alexbrainman.

Status changed to WaitingForReply.

[gopherbot]

Comment 2 by zippoxer:

I have same files and I typed the same commands shown in your comment, but I get:
$ make test
gotest
rm -f _test/a.a
CGOPKGPATH= cgo --  a.go
touch _obj/_cgo_run
8g  -o _gotest_.8  _obj/a.cgo1.go _obj/_cgo_gotypes.go
8c -FVw -IC:/Go/pkg/windows_386 -I . -o "_cgo_defun.8" _obj/_cgo_defun.c
gcc -m32 -I . -g -fPIC -O2 -o _cgo_main.o -c   _obj/_cgo_main.c
_obj/_cgo_main.c:1:0: warning: -fPIC ignored for target (all code is position in
dependent)
gcc -m32 -I . -g -fPIC -O2 -o a.cgo2.o -c   _obj/a.cgo2.c
_obj/a.cgo2.c:1:0: warning: -fPIC ignored for target (all code is position indep
endent)
gcc -m32 -I . -g -fPIC -O2 -o _cgo_export.o -c   _obj/_cgo_export.c
_obj/_cgo_export.c:1:0: warning: -fPIC ignored for target (all code is position
independent)
gcc -m32 -g -fPIC -O2 -o _cgo1_.o _cgo_main.o a.cgo2.o _cgo_export.o
cgo -dynimport _cgo1_.o >_obj/_cgo_import.c_ && mv -f _obj/_cgo_import.c_ _obj/_
cgo_import.c
8c -FVw -I . -o "_cgo_import.8" _obj/_cgo_import.c
rm -f _test/a.a
gopack grc _test/a.a _gotest_.8  _cgo_defun.8 _cgo_import.8 a.cgo2.o _cgo_export
.o
throw: runtime: cannot reserve arena virtual address space
gotest: "./8.out.exe" failed: exit status 2
make: *** [test] Error 2
===================
Maybe cgo doesn't use the right C++ compiler? (I don't really understand these things, I
just installed mingw with it's compiler).
My UAC (security settings) are turned off, btw.

[alexbrainman]

Comment 3:

I do not what your problem is. But before we do anything else, maybe you can tell us why
this error is happening. If you could change your $GOROOT/src/pkg/runtime/windows/mem.c
file like this:
diff -r 6aa111bd2d11 src/pkg/runtime/windows/mem.c
--- a/src/pkg/runtime/windows/mem.c Tue Oct 04 15:07:28 2011 +1100
+++ b/src/pkg/runtime/windows/mem.c Wed Oct 05 14:21:57 2011 +1100
@@ -55,7 +55,12 @@
        return v;
    
    // Next let the kernel choose the address.
-   return runtime·stdcall(runtime·VirtualAlloc, 4, nil, n, (uintptr)MEM_RESERVE,
(uintptr)PAGE_EXECUTE_READWRITE);
+   v = runtime·stdcall(runtime·VirtualAlloc, 4, nil, n, (uintptr)MEM_RESERVE,
(uintptr)PAGE_EXECUTE_READWRITE);
+   if(v == nil) {
+       runtime·printf("VirtualAlloc failed with errno=%d\n", runtime·getlasterror());
+       runtime·throw("VirtualAlloc");
+   }
+   return v;
 }
 
 void
Then you would need to rebuild "runtime" package, like
cd $GOROOT/src/pkg/runtime
make clean install
Then re-run your program and tell us what errno VirtualAlloc function returns. Thank you.
Alex

[gopherbot]

Comment 4 by zippoxer:

Compiling with your patch added these lines:
VirtualAlloc failed with errno=8
throw: VirtualAlloc
Hope this helps.

[alexbrainman]

Comment 5:

errno=8 is ERROR_NOT_ENOUGH_MEMORY, so it sounds like your program fails to allocate
required block of address space, perhaps, because there is not enough room to do it.
I take it, some simple program like:
package main
func main() {
        println("Hello")
}
works. Right?
Could you, please revert change to your $GOROOT/src/pkg/runtime/windows/mem.c file, and
apply this one, instead:
diff -r 6aa111bd2d11 src/pkg/runtime/windows/mem.c
--- a/src/pkg/runtime/windows/mem.c Tue Oct 04 15:07:28 2011 +1100
+++ b/src/pkg/runtime/windows/mem.c Thu Oct 06 12:21:13 2011 +1100
@@ -48,14 +48,25 @@
 void*
 runtime·SysReserve(void *v, uintptr n)
 {
+   runtime·printf("SysReserve: v=%p n=%d\n", v, n);
    // v is just a hint.
    // First try at v.
    v = runtime·stdcall(runtime·VirtualAlloc, 4, v, n, (uintptr)MEM_RESERVE, (uintptr)PAGE_EXECUTE_READWRITE);
-   if(v != nil)
+   if(v != nil) {
+       runtime·printf("SysReserve: hint worked v=%p\n", v);
        return v;
+   } else {
+       runtime·printf("SysReserve: hint failed with errno=%d\n", runtime·getlasterror());
+   }
    
    // Next let the kernel choose the address.
-   return runtime·stdcall(runtime·VirtualAlloc, 4, nil, n, (uintptr)MEM_RESERVE,
(uintptr)PAGE_EXECUTE_READWRITE);
+   v = runtime·stdcall(runtime·VirtualAlloc, 4, nil, n, (uintptr)MEM_RESERVE,
(uintptr)PAGE_EXECUTE_READWRITE);
+   if(v == nil) {
+       runtime·printf("SysReserve: accepting any address failed with errno=%d\n",
runtime·getlasterror());
+       runtime·throw("VirtualAlloc");
+   }
+   runtime·printf("SysReserve: accepting any address worked v=%p\n", v);
+   return v;
 }
 
 void
Then re-run you test again and show us the output. Please, also run little program above
and send its output too. I would like to compare the two.
I still do not know what is going on, perhaps, when you link extra mingw windows
functions, they use some extra address space, maybe they load some extra dlls, that we
have not enough room to move afterwards.
Thank you.
Alex

[gopherbot]

Comment 6 by zippoxer:

Printing "Hello" works of course, or I wasn't moving on to Windows API :P
I have 3.49GB usable memory and I successfully called Windows API with C++ (using mingw
gcc).
Okay I applied your patch and here is the output for the test:
$ make test
gotest
SysReserve: v=0x920118 n=805306368
SysReserve: hint failed with errno=487
SysReserve: accepting any address worked v=0x38020000
rm -f _test/a.a
rm -f _test/a.a
gopack grc _test/a.a _gotest_.8  _cgo_defun.8 _cgo_import.8 a.cgo2.o _cgo_export
.o
SysReserve: v=0x8fa1e0 n=805306368
SysReserve: hint failed with errno=487
SysReserve: accepting any address failed with errno=8
throw: VirtualAlloc
gotest: "./8.out.exe" failed: exit status 2
make: *** [test] Error 2
And for the little program:
$ 8.out
SysReserve: v=0x832e58 n=805306368
SysReserve: hint failed with errno=487
SysReserve: accepting any address worked v=0x38020000
Hello

[alexbrainman]

Comment 7:

I am running out of ideas. :-)
The problem here is not the amount of memory you have in your computer, but free block
of address space available in your process. Go memory manager is designed to use 1 big
block of address space (see that request for 805306368 bytes). It asks OS to reserve
single contiguous block of addresses for future use. No real memory is associated with
these addresses at the start, but will get assigned as addresses are used by the
program. The problem here is that OS refuses to allocate this 805306368 bytes block. I
don't know why. Perhaps, there is no contiguous block of such size at the time, because
memory is used by other parts of your process (dlls and such).
What I would do next is put a bug in mem.c: replace line with
"runtime·throw("VirtualAlloc");" with "*(int32*)0 = 0;". This will raise an exception
at the right moment in your program. If you run it under debugger (I use
http://www.ollydbg.de/), you should be able to see memory map of your process, this will
give you a clue if there is free block of addresses or not and why. Here is my run:
SysReserve: v=0xa00000 n=805306368
SysReserve: hint failed with errno=487
SysReserve: accepting any address worked v=0xd30000
here is my map:
Memory map
Address   Size      Owner            Section    Contains                 Type  Access  
Initial >Mapped as
00010000  00001000 >                >           Environment              Priv >RW      
RW
00020000  00001000 >                >           Process Parameters       Priv >RW      
RW
0005F000  00001000 >                >                                    Priv >RW 
Guar>RW  Guar>
00060000  00010000 >                >           Stack of main thread     Priv >RW      
RW
00070000  00006000 >                >                                    Priv >RW      
RW
00170000  00003000 >                >                                    Map  >RW      
RW
00180000  00016000 >                >                                    Map  >R       
R        C:\WINNT\system32\unicode.nls
001A0000  0002F000 >                >                                    Map  >R       
R        C:\WINNT\system32\locale.nls
001D0000  00041000 >                >                                    Map  >R       
R        C:\WINNT\system32\sortkey.nls
00220000  00004000 >                >                                    Map  >R       
R        C:\WINNT\system32\sorttbls.nls
00230000  00008000 >                >                                    Map  >R E     
R E
002F0000  00002000 >                >                                    Map  >R E     
R E
00300000  00043000 >                >           GDI handles              Map  >R        R
00400000  00001000 >8_out           >           PE header                Img  >R       
RWE Copy>
00401000  000E9000 >8_out           >.text      Code                     Img  >R E     
RWE Copy>
004EA000  00421000 >8_out           >.data      Data                     Img  >RW 
Copy>RWE Copy>
0090B000  00001000 >8_out           >/4                                  Img  >R       
RWE Copy>
0090C000  0000E000 >8_out           >/18                                 Img  >R       
RWE Copy>
0091A000  0000A000 >8_out           >/30                                 Img  >R       
RWE Copy>
00924000  0002B000 >8_out           >/43                                 Img  >R       
RWE Copy>
0094F000  00010000 >8_out           >/55                                 Img  >R       
RWE Copy>
0095F000  00003000 >8_out           >/71                                 Img  >R       
RWE Copy>
00962000  00001000 >8_out           >/87                                 Img  >R       
RWE Copy>
00963000  00001000 >8_out           >/102                                Img  >R       
RWE Copy>
00964000  00001000 >8_out           >.idata     Imports                  Img  >RW 
Copy>RWE Copy>
00965000  00001000 >8_out           >.edata     Exports                  Img  >R       
RWE Copy>
00966000  00001000 >8_out           >.symtab                             Img  >R       
RWE Copy>
00970000  000B5000 >                >                                    Map  >R E     
R E
00C70000  00001000 >                >                                    Priv >RW      
RW
00C80000  00001000 >                >                                    Priv >RW      
RW
00C90000  00001000 >                >                                    Priv >RW      
RW
00D10000  00005000 >                >                                    Priv >RW      
RW
00D20000  00002000 >                >                                    Map  >R       
R        C:\WINNT\system32\ctype.nls
77570000  00001000 >winmm           >           PE header                Img  >R       
RWE Copy>
77571000  0001F000 >winmm           >.text      Code,imports,exports     Img  >R E     
RWE Copy>
77590000  00005000 >winmm           >.data      Data                     Img  >RW 
Copy>RWE Copy>
77595000  00009000 >winmm           >.rsrc      Resources                Img  >R       
RWE Copy>
7759E000  00002000 >winmm           >.reloc     Relocations              Img  >R       
RWE Copy>
77D30000  00001000 >RPCRT4          >           PE header                Img  >R       
RWE Copy>
77D31000  00067000 >RPCRT4          >.text,.orp>Code,imports,exports     Img  >R E     
RWE Copy>
77D98000  00002000 >RPCRT4          >.data      Data                     Img  >RW      
RWE Copy>
77D9A000  00001000 >RPCRT4          >.rsrc      Resources                Img  >R       
RWE Copy>
77D9B000  00004000 >RPCRT4          >.reloc     Relocations              Img  >R       
RWE Copy>
77E10000  00001000 >USER32          >           PE header                Img  >R       
RWE Copy>
77E11000  00052000 >USER32          >.text      Code,imports,exports     Img  >R E     
RWE Copy>
77E63000  00001000 >USER32          >.data      Data                     Img  >RW      
RWE Copy>
77E64000  00008000 >USER32          >.rsrc      Resources                Img  >R       
RWE Copy>
77E6C000  00003000 >USER32          >.reloc     Relocations              Img  >R       
RWE Copy>
77F40000  00001000 >GDI32           >           PE header                Img  >R       
RWE Copy>
77F41000  00038000 >GDI32           >.text      Code,imports,exports     Img  >R E     
RWE Copy>
77F79000  00001000 >GDI32           >.data      Data                     Img  >RW      
RWE Copy>
77F7A000  00001000 >GDI32           >.rsrc      Resources                Img  >R       
RWE Copy>
77F7B000  00002000 >GDI32           >.reloc     Relocations              Img  >R       
RWE Copy>
77F80000  00001000 >ntdll           >           PE header                Img  >R       
RWE Copy>
77F81000  0004D000 >ntdll           >.text,ECOD>Code,exports             Img  >R E     
RWE Copy>
77FCE000  00003000 >ntdll           >.data      Data                     Img  >RW 
Copy>RWE Copy>
77FD1000  00001000 >ntdll           >EDATA                               Img  >RW 
Copy>RWE Copy>
77FD2000  00027000 >ntdll           >.rsrc      Resources                Img  >R       
RWE Copy>
77FF9000  00003000 >ntdll           >.reloc     Relocations              Img  >R       
RWE Copy>
78000000  00001000 >msvcrt          >           PE header                Img  >R       
RWE Copy>
78001000  00031000 >msvcrt          >.text      Code                     Img  >R E     
RWE Copy>
78032000  00008000 >msvcrt          >.rdata     Imports,exports          Img  >R       
RWE Copy>
7803A000  00007000 >msvcrt          >.data      Data                     Img  >RW 
Copy>RWE Copy>
78041000  00001000 >msvcrt          >.rsrc      Resources                Img  >R       
RWE Copy>
78042000  00003000 >msvcrt          >.reloc     Relocations              Img  >R       
RWE Copy>
7C2D0000  00001000 >ADVAPI32        >           PE header                Img  >R       
RWE Copy>
7C2D1000  0005A000 >ADVAPI32        >.text      Code,imports,exports     Img  >R E     
RWE Copy>
7C32B000  00004000 >ADVAPI32        >.data      Data                     Img  >RW 
Copy>RWE Copy>
7C32F000  00002000 >ADVAPI32        >.rsrc      Resources                Img  >R       
RWE Copy>
7C331000  00004000 >ADVAPI32        >.reloc     Relocations              Img  >R       
RWE Copy>
7C340000  00001000 >Secur32         >           PE header                Img  >R       
RWE Copy>
7C341000  0000B000 >Secur32         >.text      Code,imports,exports     Img  >R E     
RWE Copy>
7C34C000  00001000 >Secur32         >.data      Data                     Img  >RW      
RWE Copy>
7C34D000  00001000 >Secur32         >.rsrc      Resources                Img  >R       
RWE Copy>
7C34E000  00001000 >Secur32         >.reloc     Relocations              Img  >R       
RWE Copy>
7C570000  00001000 >KERNEL32        >           PE header                Img  >R       
RWE Copy>
7C571000  0005A000 >KERNEL32        >.text      Code,imports,exports     Img  >R E     
RWE Copy>
7C5CB000  00004000 >KERNEL32        >.data      Data                     Img  >RW      
RWE Copy>
7C5CF000  00051000 >KERNEL32        >.rsrc      Resources                Img  >R       
RWE Copy>
7C620000  00004000 >KERNEL32        >.reloc     Relocations              Img  >R       
RWE Copy>
7F6F0000  00007000 >                >                                    Map  >R E     
R E
7FFB0000  00024000 >                >           Code pages               Map  >R        R
7FFDE000  00001000 >                >           Data block of main threa>Priv >RWE     
RWE
7FFDF000  00001000 >                >           Process Environment Bloc>Priv >RWE     
RWE
7FFE0000  00001000 >                >           User Shared Data         Priv >R        R
As you can see, OS gave me address 0xd30000 which is right after end of program
00D20000  00002000 >                >                                    Map  >R       
R        C:\WINNT\system32\ctype.nls
and before start of first dll
77570000  00001000 >winmm           >           PE header                Img  >R       
RWE Copy>
Perhaps, you will see that there is no hole big enough to fit 805306368 bytes.
Otherwise, OS is lying when returns ERROR_NOT_ENOUGH_MEMORY and there is a different
reason for failure. Perhaps, it is security related.
Alex

[gopherbot]

Comment 8 by zippoxer:

Here is my output:
SysReserve: v=0x8fa1d0 n=805306368
SysReserve: hint failed with errno=487
SysReserve: accepting any address failed with errno=8
And my memory map:
Memory map
Address   Size      Owner            Section    Contains                 Type  Access  
Initial >Mapped as
00010000  00010000 >                >                                    Map  >RW      
RW
00020000  00010000 >                >                                    Map  >RW      
RW
0006D000  00001000 >                >                                    Priv >RW 
Guar>RW  Guar>
0006E000  00002000 >                >           Stack of main thread     Priv >RW      
RW
00070000  00004000 >                >                                    Map  >R        R
00080000  00001000 >                >                                    Priv >RW      
RW
00090000  00067000 >                >                                    Map  >R       
R        C:\Windows\System32\locale.nls
00100000  0000C000 >                >                                    Map  >R        R
001C0000  00003000 >                >                                    Map  >R        R
001D0000  00001000 >                >                                    Priv >RW      
RW
001E0000  00001000 >                >                                    Priv >RW      
RW
00200000  00003000 >                >                                    Priv >RW      
RW
00230000  00006000 >                >                                    Priv >RW      
RW
00400000  00001000 >8_out           >           PE header                Img  >R       
RWE Copy>
00401000  000E2000 >8_out           >.text      Code                     Img  >R E     
RWE Copy>
004E3000  00418000 >8_out           >.data      Data                     Img  >RW 
Copy>RWE Copy>
008FB000  00001000 >8_out           >/4                                  Img  >R       
RWE Copy>
008FC000  0000D000 >8_out           >/18                                 Img  >R       
RWE Copy>
00909000  00009000 >8_out           >/30                                 Img  >R       
RWE Copy>
00912000  00027000 >8_out           >/43                                 Img  >R       
RWE Copy>
00939000  0000F000 >8_out           >/55                                 Img  >R       
RWE Copy>
00948000  00003000 >8_out           >/71                                 Img  >R       
RWE Copy>
0094B000  00001000 >8_out           >/87                                 Img  >R       
RWE Copy>
0094C000  00001000 >8_out           >/102                                Img  >R       
RWE Copy>
0094D000  00001000 >8_out           >.idata     Imports                  Img  >RW 
Copy>RWE Copy>
0094E000  00001000 >8_out           >.edata     Exports                  Img  >R       
RWE Copy>
0094F000  00001000 >8_out           >.symtab                             Img  >R       
RWE Copy>
00950000  00101000 >                >           GDI handles              Map  >R        R
00A60000  00108000 >                >                                    Map  >R        R
0DCE0000  00001000 >KERNELBASE      >           PE header                Img  >R       
RWE Copy>
0DCE1000  00043000 >KERNELBASE      >.text      Code,imports,exports     Img  >R E     
RWE Copy>
0DD24000  00002000 >KERNELBASE      >.data      Data                     Img  >RW      
RWE Copy>
0DD26000  00001000 >KERNELBASE      >.rsrc      Resources                Img  >R       
RWE Copy>
0DD27000  00003000 >KERNELBASE      >.reloc     Relocations              Img  >R       
RWE Copy>
38010000  00001000 >                >                                    Img  >R       
RWE Copy>
402C0000  00001000 >LPK             >           PE header                Img  >R       
RWE Copy>
402C1000  00006000 >LPK             >.text      Code,imports,exports     Img  >R E     
RWE Copy>
402C7000  00001000 >LPK             >.data      Data                     Img  >RW      
RWE Copy>
402C8000  00001000 >LPK             >.rsrc      Resources                Img  >R       
RWE Copy>
402C9000  00001000 >LPK             >.reloc     Relocations              Img  >R       
RWE Copy>
41840000  00001000 >IMM32           >           PE header                Img  >R       
RWE Copy>
41841000  00017000 >IMM32           >.text      Code,imports,exports     Img  >R E     
RWE Copy>
41858000  00001000 >IMM32           >.data      Data                     Img  >RW      
RWE Copy>
41859000  00005000 >IMM32           >.rsrc      Resources                Img  >R       
RWE Copy>
4185E000  00001000 >IMM32           >.reloc     Relocations              Img  >R       
RWE Copy>
6F8E0000  00001000 >USP10           >           PE header                Img  >R       
RWE Copy>
6F8E1000  0005B000 >USP10           >.text      Code,imports,exports     Img  >R E     
RWE Copy>
6F93C000  00002000 >USP10           >.data      Data                     Img  >RW      
RWE Copy>
6F93E000  0002A000 >USP10           >Shared                              Img  >R       
RWE Copy>
6F968000  00012000 >USP10           >.rsrc      Resources                Img  >R       
RWE Copy>
6F97A000  00003000 >USP10           >.reloc     Relocations              Img  >R       
RWE Copy>
6FF50000  00001000 >msvcrt          >           PE header                Img  >R       
RWE Copy>
6FF51000  0009F000 >msvcrt          >.text      Code,imports,exports     Img  >R E     
RWE Copy>
6FFF0000  00007000 >msvcrt          >.data      Data                     Img  >RW 
Copy>RWE Copy>
6FFF7000  00001000 >msvcrt          >.rsrc      Resources                Img  >R       
RWE Copy>
6FFF8000  00004000 >msvcrt          >.reloc     Relocations              Img  >R       
RWE Copy>
70990000  00001000 >MSCTF           >           PE header                Img  >R       
RWE Copy>
70991000  00083000 >MSCTF           >.text      Code,imports,exports     Img  >R E     
RWE Copy>
70A14000  00002000 >MSCTF           >.data      Data                     Img  >RW 
Copy>RWE Copy>
70A16000  00041000 >MSCTF           >.rsrc      Resources                Img  >R       
RWE Copy>
70A57000  00005000 >MSCTF           >.reloc     Relocations              Img  >R       
RWE Copy>
77B60000  00001000 >GDI32           >           PE header                Img  >R       
RWE Copy>
77B61000  00048000 >GDI32           >.text      Code,imports,exports     Img  >R E     
RWE Copy>
77BA9000  00002000 >GDI32           >.data      Data                     Img  >RW      
RWE Copy>
77BAB000  00001000 >GDI32           >.rsrc      Resources                Img  >R       
RWE Copy>
77BAC000  00002000 >GDI32           >.reloc     Relocations              Img  >R       
RWE Copy>
77D10000  00001000 >user32          >           PE header                Img  >R       
RWE Copy>
77D11000  00068000 >user32          >.text      Code,imports,exports     Img  >R E     
RWE Copy>
77D79000  00001000 >user32          >.data      Data                     Img  >RW      
RWE Copy>
77D7A000  0005B000 >user32          >.rsrc      Resources                Img  >R       
RWE Copy>
77DD5000  00004000 >user32          >.reloc     Relocations              Img  >R       
RWE Copy>
77DE0000  00001000 >kernel32        >           PE header                Img  >R       
RWE Copy>
77DE1000  000C5000 >kernel32        >.text      Code,imports,exports     Img  >R E     
RWE Copy>
77EA6000  00001000 >kernel32        >.data      Data                     Img  >RW      
RWE Copy>
77EA7000  00001000 >kernel32        >.rsrc      Resources                Img  >R       
RWE Copy>
77EA8000  0000C000 >kernel32        >.reloc     Relocations              Img  >R       
RWE Copy>
77EC0000  00001000 >ntdll           >           PE header                Img  >R       
RWE Copy>
77EC1000  000D5000 >ntdll           >.text      Code,exports             Img  >R E     
RWE Copy>
77F96000  00001000 >ntdll           >RT                                  Img  >R E     
RWE Copy>
77F97000  00009000 >ntdll           >.data      Data                     Img  >RW 
Copy>RWE Copy>
77FA0000  00057000 >ntdll           >.rsrc      Resources                Img  >R       
RWE Copy>
77FF7000  00005000 >ntdll           >.reloc     Relocations              Img  >R       
RWE Copy>
7F6F0000  00005000 >                >                                    Map  >R        R
7FFB0000  00023000 >                >           Code pages               Map  >R        R
7FFD6000  00001000 >                >           Process Environment Bloc>Priv >RW      
RW
7FFDF000  00001000 >                >           Data block of main threa>Priv >RW      
RW
7FFE0000  00001000 >                >           User Shared Data         Priv >R        R
80000000  7FFF0000 >                >           Kernel memory            Kern >
So you are right :P
The program ends at 0xB68000 (0xA60000 + 0x108000) and the first dll starts at
0xDCE0000. The size of the hole is 219643904 (smaller than 805306368) bytes.
Why? :/

[alexbrainman]

Comment 9:

I do not know how KERNELBASE.DLL happened to be in the middle of your address space. And
I don't know how to control it. When I run your process on all computers I have: XP,
W2K, Windows 7; none of them have KERNELBASE.DLL loaded when I stop process in
runtime·SysReserve as you did. Perhaps, someone else will suggest something new.

Labels changed: added os-windows.

Status changed to Accepted.

[gopherbot]

Comment 10 by zippoxer:

I just tried running a go program called Pokemon Universe (http://pokemon-universe.com/)
and got the same error (throw: runtime: cannot reserve arena virtual address space).
This game worked not long time ago, on my old computer.
So this might be a problem with my current windows installation - it can't run go
programs, or more exactly, go programs using cgo.
I'm only worried because so many other applications are working perfectly. So part of
the problem is with go and the other is with my windows installation.

[alexbrainman]

Comment 11:

I think the problem you are seeing is because KERNELBASE.DLL happend to be loaded in the
middle of your address space. Your address space got fragmented, so go runtime can't
allocate this big chunk of memory it uses for garbage collector.
Non-cgo programs shouldn't be affected, because go runtime loads most of its dlls after
this big block of memory is allocated. So Windows just find another place for all these
dlls.
Cgo programs, on the other hand, have dlls loaded during process load time, even before
process started running. Dlls are listed in the pe executable file header. And Windows
process loader just loads them all alongside with the executable. Loader picks whatever
addresses it likes. This makes your address fragmented even before your process starts.
I am surprised, no-one reported anything like that before.
If nothing else, we would have to "reserve" this big contiguous block of addresses
somehow in our pe file description. Maybe create a special pe section for it, or include
it as part of existing BSS section.
Alex

[gopherbot]

Comment 12 by zippoxer:

Or to reinstall my windows. I tried adding a section to the PE file with the size of
805306368, but windows complained about validity. I'm (almost) sure I will be able to
execute cgo programs with the right person editing the PE file, but after every
compilation?
So I will reinstall windows soon, unless someone resists.
And if the problem ever appears again, I will know if it's a software that I installed
or a windows setting that I changed.
Proper cgo programs ran successfully in so many windows installations (I take Pokemon
Universe as an example) and nobody complained about "throw: runtime: cannot reserve
arena virtual address space", so what's the chance that this problem will appear again?
Thank you Alex for exploring the problem (until actually discovering it). Seriously,
without your help I was quitting go (until the world moves unix or until I format my
windows and accidentally trying a cgo program).

[alexbrainman]

Comment 13:

Adding big pe section is not enough (even if you do it properly). You also need to make
go use it during runtime.
I have a feeling we will hear more about your problem. Lets keep this issue opened.

[rsc]

Comment 14:

Labels changed: added priority-later, removed priority-medium.

[rsc]

Comment 15:

Labels changed: added priority-go1.