Unexpected different behavior with append on a byte slice vs int slice.

[SpicyLemon]

What version of Go are you using (go version)?

$ go version
go version go1.15.8 darwin/amd64

Does this issue reproduce with the latest release?

Unsure. It is reproducible in the go playground though: https://play.golang.org/p/O6ZPNn4FN6D

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="$HOME/Library/Caches/go-build"
GOENV="$HOME/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="$HOME/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="$HOME/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.15.8/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.15.8/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/px/yz_xp7wj195g7n9k2h4dxp8w0000gp/T/go-build808974878=/tmp/go-build -gno-record-gcc-switches -fno-common"

Note: In the above, I manually updated the output to mask my home directory by replacing it with $HOME. The original output of go env had the actual paths.

What did you do?

1. Create a []byte with a single initial value.
2. Use append to add another value to it.
3. Use append to create a new []byte containing the values from 1 and 2 plus an extra value.
4. Use append on the 1st slice to create another new []byte containing the values from 1 and 2 plus three extra values.

At this point, the last entry in the slice created in step 3 has changed to the first new byte value provided during step 4.

Doing the same process with an []int does not cause this change.

package main

import (
	"fmt"
)

func main() {
	b := []byte{0x01}
	i := []int{1}

	fmt.Printf("Initial state.\n")
	fmt.Printf("b: %v\n", b)
	fmt.Printf("i: %v\n", i)

	fmt.Printf("Adding 2 to each and assigning back to original.\n")
	b = append(b, 0x02)
	i = append(i, 2)
	fmt.Printf("b: %v\n", b)
	fmt.Printf("i: %v\n", i)

	fmt.Printf("Adding 3 to each and assigning to new variable.\n")
	c := append(b, 0x03)
	j := append(i, 3)
	fmt.Printf("b: %v\n", b)
	fmt.Printf("i: %v\n", i)
	fmt.Printf("c: %v\n", c)
	fmt.Printf("j: %v\n", j)

	fmt.Printf("Adding 4, 5, 6 to each and assigning to new another new variable.\n")
	d := append(b, 0x04, 0x05, 0x06)
	k := append(i, 4, 5, 6)
	fmt.Printf("b: %v\n", b)
	fmt.Printf("i: %v\n", i)
	fmt.Printf("c: %v <-- look here\n", c)
	fmt.Printf("j: %v\n", j)
	fmt.Printf("d: %v\n", d)
	fmt.Printf("k: %v\n", k)
}

What did you expect to see?

Initial state.
b: [1]
i: [1]
Adding 2 to each and assigning back to original.
b: [1 2]
i: [1 2]
Adding 3 to each and assigning to new variable.
b: [1 2]
i: [1 2]
c: [1 2 3]
j: [1 2 3]
Adding 4, 5, 6 to each and assigning to new another new variable.
b: [1 2]
i: [1 2]
c: [1 2 3] <-- look here
j: [1 2 3]
d: [1 2 4 5 6]
k: [1 2 4 5 6]

What did you see instead?

Initial state.
b: [1]
i: [1]
Adding 2 to each and assigning back to original.
b: [1 2]
i: [1 2]
Adding 3 to each and assigning to new variable.
b: [1 2]
i: [1 2]
c: [1 2 3]
j: [1 2 3]
Adding 4, 5, 6 to each and assigning to new another new variable.
b: [1 2]
i: [1 2]
c: [1 2 4] <-- look here
j: [1 2 3]
d: [1 2 4 5 6]
k: [1 2 4 5 6]

Extra notes:

If you remove the "Adding 2 to each and assigning back to original." section, the output is as expected:

new output

Initial state.
b: [1]
i: [1]
Adding 3 to each and assigning to new variable.
b: [1]
i: [1]
c: [1 3]
j: [1 3]
Adding 4, 5, 6 to each and assigning to new another new variable.
b: [1]
i: [1]
c: [1 3] <-- look here
j: [1 3]
d: [1 4 5 6]
k: [1 4 5 6]

[randall77]

This is how slices and append work. https://blog.golang.org/slices-intro

The backing store for b, c, and d, are all the same underlying array. d := append(b, 0x04, 0x05, 0x06) modifies the 3rd element of that backing store, which is what c points to.
For the integer slice, append grows the backing store on the k := append(i, 4, 5, 6), so the same effect doesn't happen.

This is not a bug in Go. Please reopen if you disagree.

[SpicyLemon]

I disagree, but don't seem to have the power to reopen a closed issue.

The same thing that is done to create c and d is being done to create j and k. The only difference is the the former are []byte and the latter are []int. The behavior of append should be the same, but it's not. If append is supposed to modify the underlying byte array, then it should also be modifying the underlying int array.

According to A Tour of Go: Appending to a slice:

If the backing array of s is too small to fit all the given values a bigger array will be allocated. The returned slice will point to the newly allocated array.

That's why, in order to properly append a value, you have to do s := append(s, v) rather than being able to just call append(s, v) and letting append update s as needed.

[ianlancetaylor]

The relevant code here is

    c := append(b, 0x03)
    d := append(b, 0x04, 0x05, 0x06)
    fmt.Printf("c: %v <-- look here\n", c)

The assignment to c may or may not allocate a new backing slice. The assignment to d may or may not allocate a new backing slice. If neither assignment allocates a new backing slice, then the append used to initialize d will overwrite the 3 appended to c. If either assignment allocates a new backing slice, c and d will be independent, and the append used to initialize d will not overwrite the 3.

Whether an append allocates a new backing slice depends on the size of the memory already allocated and the size of the memory needed. Those sizes are different for byte and int. There is no reason to think that they will behave the same in all cases.

This is not a bug. This is how slices work.

[SpicyLemon]

Okay. I think I found the source of my confusion. The append function increases capacity (and changes to a new underlying array) differently for different types. And whether or not to grow depends on the capacity instead of the length.