gccgo: cgo: false positive cgo argument has Go pointer to Go pointer

[tamird]

What version of Go are you using (go version)?

go version go1.9 gccgo (GCC) 8.0.0 20180108 (experimental) linux/amd64

Description

The following code exists in CockroachDB:
https://github.com/cockroachdb/cockroach/blob/4e6ec545cc65dc0051a797996e9642b4b5e8bdc2/pkg/storage/engine/rocksdb.go#L447-L450

https://github.com/cockroachdb/cockroach/blob/4e6ec545cc65dc0051a797996e9642b4b5e8bdc2/pkg/storage/engine/rocksdb.go#L584

https://github.com/cockroachdb/cockroach/blob/4e6ec545cc65dc0051a797996e9642b4b5e8bdc2/c-deps/libroach/db.cc#L1616

Said plainly, initialization across cgo is done by passing a Go **C.DBEngine which points to nil to a C function that allocates and sets the pointed-to *C.DBEngine. This code works perfectly when compiled with gc, but produces this error when compiled with gccgo:

panic: runtime error: cgo argument has Go pointer to Go pointer

goroutine 24 [running]:
panic
        ../../../src/libgo/go/runtime/panic.go:543
github_com_cockroachdb_cockroach_pkg_storage_engine._cgoCheckPointer
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/cgo-c-prolog-gccgo:74
engine.$nested21
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:584
github_com_cockroachdb_cockroach_pkg_storage_engine.open.pN59_github_com_cockroachdb_cockroach_pkg_storage_engine.RocksDB
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:584
engine.newMemRocksDB
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:518
github_com_cockroachdb_cockroach_pkg_storage_engine.NewInMem
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/in_mem.go:37
github_com_cockroachdb_cockroach_pkg_server.CreateEngines.pN50_github_com_cockroachdb_cockroach_pkg_server.Config
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/server/config.go:483
github_com_cockroachdb_cockroach_pkg_server.Start.pN50_github_com_cockroachdb_cockroach_pkg_server.Server
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:858
cli.$nested72
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/cli/start.go:518
cli.$nested70
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/cli/start.go:500
created by cli.runStart
        /usr/local/home/tduberstein/go/src/github.com/cockroachdb/cockroach/pkg/cli/start.go:474 +3255

According to https://golang.org/cmd/cgo/#hdr-Passing_pointers:

Go code may pass a Go pointer to C provided the Go memory to which it points does not contain any Go pointers.

It's not exactly clear if nil Go pointers are permitted, but the gc implementation certainly permits it. Perhaps gccgo is missing a null check?

@ianlancetaylor

[ianlancetaylor]

Can you show us a way to reproduce the problem locally?

[tamird]

Oddly, I can't reproduce this after rebasing my gccgo work on CockroachDB's latest master branch.

Nominally, the repro instructions are roughly the same as #23393 (comment), except an on-disk store must be used (./cockroach start --insecure instead of ./cockroach start --insecure -s type=mem,size=1GiB).

[tamird]

Ah, this reproduces occasionally using

cd $(go env GOPATH)/src/github.com/cockroachdb/cockroach
make stress PKG=./pkg/cli TESTS=TestZip

when go is gccgo (note the use of stress above, it takes a few runs).

[tamird]

@thanm care to look into this one, since you already have the necessary local setup?

[thanm]

I'll take a look later this morning

[tamird]

Did a bit of digging under gdb:

(gdb) bt
#0  runtime.inHeapOrStack (b=842356777072, b@entry=140737205361048) at ../../../src/libgo/go/runtime/mheap.go:418
#1  runtime.cgoIsGoPointer (p=p@entry=0xc420606070) at ../../../src/libgo/go/runtime/cgocall.go:272
#2  0x00007ffff6d02a30 in runtime.cgoIsGoPointer (p=0xc420606070) at ../../../src/libgo/go/runtime/cgocall.go:268
#3  runtime.cgoCheckPointer (ptr=..., args=...) at ../../../src/libgo/go/runtime/cgocall.go:64
#4  0x0000000000f7ab07 in github_com_cockroachdb_cockroach_pkg_storage_engine._cgoCheckPointer (ptr=..., args=...) at cgo-c-prolog-gccgo:74
#5  0x0000000000f3ee19 in engine.func1 (_cgo0=0xc420606070, _cgo1=..., _cgo2=...)
    at /home/tduberstein/local/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:584
#6  0x0000000000f3ea4e in github_com_cockroachdb_cockroach_pkg_storage_engine.RocksDB.open (r=0xc420606000)
    at /home/tduberstein/local/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:584
#7  0x0000000000f3df89 in engine.newMemRocksDB (attrs=..., param=..., MaxSizeBytes=536870912)
    at /home/tduberstein/local/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:518
#8  0x0000000000e31e9a in github_com_cockroachdb_cockroach_pkg_storage_engine.NewInMem (attrs=..., cacheSize=1048576)
    at /home/tduberstein/local/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/in_mem.go:37
#9  0x0000000000e501aa in engine.testBatchBasics (t=0xc420b1e0f0, writeOnly=false,
    commit=0x1e43840 <github_com_cockroachdb_cockroach_pkg_storage_engine.TestBatchBasics..func1..f>)
    at /home/tduberstein/local/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/batch_test.go:56
#10 0x0000000000e5296a in github_com_cockroachdb_cockroach_pkg_storage_engine.TestBatchBasics (t=0xc420b1e0f0)
    at /home/tduberstein/local/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/batch_test.go:132
#11 0x00007ffff6d914f0 in testing.tRunner (param=<optimized out>, fn=<optimized out>) at ../../../src/libgo/go/testing/testing.go:777
#12 0x00007ffff6e21d25 in __morestack () at ../../../src/libgcc/config/i386/morestack.S:546
#13 0x00007ffff6d915eb in testing.testing..thunk24 (__go_thunk_parameter=<optimized out>) at ../../../src/libgo/go/testing/testing.go:824
#14 0x00007ffff6d19ea9 in runtime.kickoff () at ../../../src/libgo/go/runtime/proc.go:1161
#15 0x00007ffff59f2d40 in ?? () from /lib64/libc.so.6
#16 0x0000000000000000 in ?? ()

The interesting stuff is happening in frame 6 (Go function passing a pointer) and frames 3 and below (Go functions checking that pointer).

(gdb) frame 6
#6  0x0000000000f3ea4e in github_com_cockroachdb_cockroach_pkg_storage_engine.RocksDB.open (r=0xc420606000)
    at /home/tduberstein/local/go/src/github.com/cockroachdb/cockroach/pkg/storage/engine/rocksdb.go:584
584             status := C.DBOpen(&r.rdb, goToCSlice([]byte(r.cfg.Dir)),
(gdb) print r.rdb
$6 = (.github.com/cockroachdb/cockroach/pkg/storage/engine._Ctype_struct_DBEngine *) 0x0

So we are passing a pointer to a null pointer.

(gdb) frame 3
#3  runtime.cgoCheckPointer (ptr=..., args=...) at ../../../src/libgo/go/runtime/cgocall.go:64
64                      if !cgoIsGoPointer(p) {
(gdb) list
59              if len(args) > 0 && (t.kind&kindMask == kindPtr || t.kind&kindMask == kindUnsafePointer) {
60                      p := ep.data
61                      if t.kind&kindDirectIface == 0 {
62                              p = *(*unsafe.Pointer)(p)
63                      }
64                      if !cgoIsGoPointer(p) {
65                              return
66                      }
67                      aep := (*eface)(unsafe.Pointer(&args[0]))
68                      switch aep._type.kind & kindMask {
(gdb) print ep.data
$10 = (Pointer) 0xc420606070
(gdb) print p
$11 = (void *) 0xc420606070
(gdb) print t.kind&(1<<5) == 0
$14 = 0
(gdb) x 0xc420606070
0xc420606070:   0x00000000

This is weird. We're skipping the dereference p = *(*unsafe.Pointer)(p), so we're going to check the **C.DBEngine instead of the *C.DBEngine, and that causes the failure. As expected, the memory pointed to by the **C.DBEngine is nil.

So, somehow the **C.DBEngine got compiled into a kindDirectIface type so the compiler thinks "the data pointer field in the interface value is exactly the value stored in the interface". That same comment goes on:

// In the current gccgo implementation, this is set
// only for pointer types (including unsafe.Pointer). In the future it
// could also be set for other types: channels, maps, functions,
// single-field structs and single-element arrays whose single field
// is simply a pointer.

In other words, gccgo is setting kindDirectIface for this type, which appears to be in violation of assumptions made when writing cgoCheckPointer (https://golang.org/cl/16003).

[ianlancetaylor]

Where you print t.kind, can you print all of t, and also *t.string? Thanks.

[tamird]

(gdb) print t
$1 = (.runtime._type *) 0x1e56d60 <type...1.1github_com_cockroachdb_cockroach_pkg_storage_engine._Ctype_struct_DBEngine>
(gdb) print *t.string
$2 = 0x1e25f78 "**\tgithub_com_cockroachdb_cockroach_pkg_storage_engine\tengine._Ctype_struct_DBEngine"

[ianlancetaylor]

Thanks. I'm not seeing what is wrong here. It's correct that kindDirectIface is set. It's correct that it is looking at p == &r.rdb. At this point it's determining, correctly, that p is a Go pointer. It's OK to pass a Go pointer to a cgo call. What is not OK is passing a Go pointer that points to a Go pointer. So the question is: what happens next? Somewhere after this it decides that p points to memory that contains a Go pointer. What are the steps that get there?

[tamird]

Somewhere after this it decides that p points to memory that contains a Go pointer. What are the steps that get there?

My reading of the code is that it decides that p points to Go memory. In other words, this dereference is intended to grab the pointed-to memory, but because it is skipped, the subsequent call to cgoIsGoPointer correctly returns true.

https://github.com/golang/gofrontend/blob/65eaa9003db4effc9c5ffe9c955e9534ba5d7d15/libgo/go/runtime/cgocall.go#L262-L289

[ianlancetaylor]

In this case, p, which has been passed as &r.rdb, does point to Go memory. You are saying that that is wrong, but as far as I can see it is right. It's not a problem for p to point to Go memory. It's only a problem if the memory to which p points itself contains a pointer to Go memory.

[ianlancetaylor]

To put it another way, where is the error reported? It's not reported by the code you discussed above.

[tamird]

I believe from there it's https://github.com/golang/gofrontend/blob/65eaa9003db4effc9c5ffe9c955e9534ba5d7d15/libgo/go/runtime/cgocall.go#L93 to https://github.com/golang/gofrontend/blob/65eaa9003db4effc9c5ffe9c955e9534ba5d7d15/libgo/go/runtime/cgocall.go#L206 to https://github.com/golang/gofrontend/blob/65eaa9003db4effc9c5ffe9c955e9534ba5d7d15/libgo/go/runtime/cgocall.go#L238, though I haven't traced through this in gdb yet.