reflect: Value.Addr().Elem() loses flagEmbedRO in origin value

[kezhuw]

This makes exported fields in unexported anonymous field not settable

What version of Go are you using (go version)?

$ go version
go version go1.12.6 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.6/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/b6/81wsy1517sxdpkd09pwj98_w0000gn/T/go-build799570696=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

https://play.golang.org/p/mKRa1-3j2He

package main

import (
	"fmt"
	"reflect"
)

type embed struct{ X int }

type S1 struct {
	embed
}

func printCanSet(value reflect.Value, index []int) {
	current := value
	for _, i := range index {
		if i == -1 {
		        // Addr() decays flagRO, which is a combination of flagEmbedRO and flagStickyRO, to flagStickyRO which is propagated to
			// all nested fields, thus making exported fields in unexported anonymous field not settable.
			//
			// See for implementation details:
			// * https://github.com/golang/go/blob/master/src/reflect/value.go#L83
			// * https://github.com/golang/go/blob/master/src/reflect/value.go#L256
			// * https://github.com/golang/go/blob/master/src/reflect/value.go#L829
			current = current.Addr().Elem()
		} else {
			current = current.Field(i)
		}
	}
	fmt.Printf("Value %+v lookup index %+v: CanSet ? %t\n", value, index, current.CanSet())
}

func main() {
	value := reflect.ValueOf(&S1{}).Elem()
	printCanSet(value, []int{0, 0})
	printCanSet(value, []int{0, -1, 0})
}

What did you expect to see?

The second should prints true for CanSet.

What did you see instead?

The second prints false for CanSet.

See also

• 6471ace#diff-299545bcc665f75feeed30dce79b50ecR243 introduced this behavior.
• I have present a trial fix in https://github.com/kezhuw/go/commits/fix-reflect-addr-lost-embed-ro, it contains two commits:
  • The first commit (kezhuw/go@f403764) introduces changes to break TestCanSetField in go.
  • The second commit (kezhuw/go@376b342) fixes breaking tests caused by previous commit.

[ianlancetaylor]

I don't really see how to fix this safely: ensuring that the Addr can not be used in some way to set the unexported fields. If you think that your changes are safe, do you want to send a pull request? Thanks.

[kezhuw]

@ianlancetaylor I thought after your response, I think it is safe for reasons:

• Result of Value.Addr() is not CanAddr and hence not CanSet, so the field itself can't be modified later, this does not interfere with flagRO.
• Addr().Elem() get flagRO same as origin value, so it should behave same as before in CanSet.
• Value.Field() inherits flagStickyRO but not flagEmbedRO from wrapped value, so
  • if the value itself is flagEmbedRO, Value.Field()'s flagRO depends only on that field but not its wrapped value;
  • if the value itself is flagStickyRO, all its fields are unexported.

I have update commits with more test cases in trial fix branch to (kezhuw/go@306e9c9 kezhuw/go@a16093d), I plan to send a pull request later after job (at least 10 hours late) if above sounds good to you.

See Value.Field().

[gopherbot]

Change https://golang.org/cl/183937 mentions this issue: Fix lost flagEmbedRO in reflect.Value.Addr().Elem()

[mdempsky]

I'm sympathetic to the rationale that if v.CanAddr(), then v.Addr().Elem() should be the same as just v alone. After all, if x is an addressable Go expression, then *&x is the same as x.

In general, package reflect exposes the same API as you could use through a package's exported interface. An issue arises when fields are promoted from a non-exported embedded field. For example:

type t struct { X int }
type U struct { t }
var V U

An external package can access V.X, which is short-hand for V.t.X (though the latter isn't actually in external packages).

This is problematic for package reflect because reflect doesn't provide a way to directly access promoted fields. They have to be accessed field-by-field, and the behavior differed between cmd/compile (née gc) and gccgo. It was decided in #12367 (included in Go 1.6) to reconcile this by special casing accesses like v.Field(0).Field(0) if the result is an exported field. That is, to special case V.t.X, since package reflect doesn't support V.X directly. We also special case dereferences, because of pointer embedding. (This was further fixed in #22031, released in Go 1.10.)

This is where the intuition about *& falls down in my opinion: is that intuition important enough to further special case expressions (*&V.t).X? I don't think it's unreasonable, but I'm also not convinced.

@kezhuw You mentioned finding this when upgrading from Go 1.5 to Go 1.11. Do you mean this broke some code you had written? Can you share the code?