encoding/json Unmarshal function does not work with lowercase struct fields of go type

[ogeffert]

What version of Go are you using (go version)?

$ go version
go version go1.20.4 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/home/otfried/bin"
GOCACHE="/home/otfried/.cache/go-build"
GOENV="/home/otfried/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/otfried/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/otfried/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.4"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="0"
GOMOD="/home/otfried/gospace/rg6server/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2527071798=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
"encoding/json"
"fmt"
)

func main() {
var bytes = []byte({"user": "magnus", "rating": "3856"})
type Animal struct {
user string
rating string
}
var one_animal Animal
err := json.Unmarshal(bytes, &one_animal)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v", one_animal)
}

What did you expect to see?

{User:magnus Rating:3856}

What did you see instead?

{user: rating:}

[ogeffert]

I took the Unmarshal example from the library und slowly changed it to become closer to what I program --- a chess server, where I tried Unmarshal but where it never worked correctly.
Then I found that the struct fields have to be User and Rating in this type declaration - instead of user and rating.
So struct fields starting with lowercase letters do NOT work with this Unmarshal function.

[ayang64]

This is not a bug but a result of the fact that unexported identifiers are not visible to other packages.

The json package needs to be able to inspect those fields in order to assign them.
The only way to make this work is by making the first character a unicode upper case letter.

https://go.dev/ref/spec#Exported_identifiers

Did that make sense?

[ogeffert]

I see what you mean and I almost anticipated it, but
when you want to save the state of a program to a json file, then you also want to save all contents of all variables - wether exported or not. So I think the upper case convention here is well meant, but it hinders a broader usage.

[ogeffert]

Thanks for looking at this.
So I guess I will have to take the sources and try to fix this for my project, ----if I find the right spot, where to do it.

Do you know the proper place ? I looked at decode.go, but apart from "export" i have no hint what to look for.

[ayang64]

@ogeffert -- I should have given a more complete answer.

You can annotate the your struct members to tell the decoder which JSON keys to map to it.

For example, your Animal struct can be annotated as follows:

type Animal struct {
  User string   `json:"user"`
  Rating string `json:"rating"`
}

Please check out the JSON and Go blog post on the main site for details on the json package.

Did that solve your problem?

[seankhliao]

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions

[ogeffert]

@seankhliao This is not a question, sorry.
It is a behaviour of the JSON package, that makes it unsuitable for certain usages
and it is - at least - undocumented.
And I see it as a bug.

[ogeffert]

@ayang64 I cannot see that annotation cures something here.
The struct fields still have to be uppercase for Unmarshal() to work.

This is a limitation that could be lifted by some boolean switch built into the library.

[ianlancetaylor]

@ogeffert This is documented at https://pkg.go.dev/encoding/json#Marshal, which says "Each exported struct field becomes a member of the object". The same applies to Unmarshal. The Marshal doc also describes how to set the JSON name when it should be distinct from the field name. You may find it helpful to read https://go.dev/blog/json.

The difference between exported and unexported fields is a characteristic of the language itself, not something that can be addressed in the library.

[ogeffert]

The language go is as it is. I like it.
However JSON is a general format and to tie it so strongly to GO specifics is a pity.

Will use the ojg package instead, as it is more liberal in its approach.
I grew up in the sixties, I need some freedom.

Thanks to all of for your comments !