proposal: Go 2: interfaces with methods which return interface{} should be fulfilled with any return type

[Bjohnson131]

Would you consider yourself a novice, intermediate, or experienced Go programmer?

Novice-Intermediate

What other languages do you have experience with?

I don't keep a running list. If I were to name my top 3, C++ VHDL, Java

Would this change make Go easier or harder to learn, and why?

Easier, it would allow users to write interfaces which form contracts with the compiler, while keeping the flexibility of interface{}

Has this idea, or one like it, been proposed before?

Yes, though the similarity is superficial.

If so, how does this proposal differ?

The other proposals are about interface{} as a data type, this is about interface as used in a signature

Who does this proposal help, and why?

It helps those who want to abstract their code, while maintaining rigidity where needed. See also the response to question 3.

Is this change backward compatible?

Yes

What is the cost of this proposal? (Every language change has a cost).

The time of implementation, and the time of the users to learn the feature. Maybe some angry programmers who disagree with the change.

How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?

govet, and some parts of lint, as well as the compiler.

What is the compile time cost?

<0 ms. Compilation would be faster.

What is the run time cost?

There is 0 runtime cost. This is an idiomatic change.

Can you describe a possible implementation?

yes.

Do you have a prototype? (This is not required.)

no

Orthogonality: how does this change interact or overlap with existing features?

It would allow users to define more rigid funcs that satisfy extremely abstract interfaces.

Is the goal of this change a performance improvement?

no.

Does this affect error handling?

only in the compiler.

Is this about generics?

only if interface{} is a generic.

What is the proposed change?

interfaces with methods which take or return interface{} should be fulfilled with any return type

Please describe as precisely as possible the change to the language.

A struct with methods which would satisfy an interface if any combination of any combination of method's return type's were changed to interface{} should satisfy the interface in question.

What would change in the language spec?

• An interface with functions that specify interface{} as a return type will be satisfied with any specified return type.

Please also describe the change informally, as in a class teaching Go.

you can return anything to satisfy an interface expecting an interface{} as a return type.

Show example code before and after the change.

type myInterface interface {
   myInterfaceFunc (input interface{}) string
   myOtherFunc (input string) interface{}
}
type myStruct struct{}
func (s myStruct) myInterfaceFunc(input interface{}) string{
   return "input"
}
func (s myStruct) myOtherFunc(input string) string{
   return input
}

With the proposed change, myStruct would satisfy the "myInterface" interface. currently it does not, as myOtherFunc returns a string, and not an interface{}

How would the language spec change?

See above. duplicate.

[ianlancetaylor]

I don't understand how this can be implemented. Suppose myInterface and myStruct are defined in two different packages. Suppose a third package takes a value of type interface{} and converts it to myInterface. Suppose the dynamic type of the interface{} value is myStruct, a type that the third package knows nothing about. In order to make this work, at the point of the interface conversion we need a method that returns a value of type interface{}, but we only have a method that returns a value of type string. Somewhere, somehow, we need some code to convert the result from string to interface{}. Where is that code created, and how?

[Bjohnson131]

Somewhere, somehow, we need some code to convert the result from string to interface{}. Where is that code created, and how?

I'm not sure of the problem. Can you elaborate @ianlancetaylor? Ideally, it would be the same code in the compiler that handles

func example() interface{} {
  return "conversion to interface{} happens here, and is currently valid Golang!"
}

[ianlancetaylor]

In that code the compiler can see clearly that it needs to convert a string to interface{}.

package a

type I interface { M() interface{} }

package b

type S struct {}
func (S) M() string { return "M" }

package c

import "b"

func F() interface{} { return b.S{} }

package d

import (
    "a"
    "c"
)

func F() {
    s := c.F()
    // Now s is type interface{}, with dynamic type b.S.
    // Note that this package knows nothing about b.
    // Now we are going to convert the b.S value to a.I.
    // Under this proposal this conversion should work
    // since b.S has method "M() string"
    // and a.I has method "M() interface{}".
    x := s.(a.I)
    // Now x is a value with a method "M() interface{}".
    // But s is a value with a method "M() string".
    // How do we change "M() string" to "M interface{}"?
    // That requires new code; where does that code live?
    // What creates it?  The compiler never sees that any
    // conversion from "string" to "interface{}" is required,
    // because it doesn't know the dynamic type of s.
}

[Bjohnson131]

In that code the compiler can see clearly that it needs to convert a string to interface{}.

I'm not sure, if you run that code, you should get
./d.go:15:10: type I is not an expression

as attempting to assign a value to a type is not permitted in GoLang.

[ianlancetaylor]

Are you sure you copied the code directly? There is no attempt to use I as an expression. The code in d.go is using a.I as a type.

But you're right, I wrote d.go incorrectly, because it is missing a type assertion. I've updated the comment above. The program then compiles. When I run it with today's compiler, I get

panic: interface conversion: b.S is not a.I: missing method M

goroutine 1 [running]:
m/d.F(...)
	/tmp/x/d/d.go:16
main.main()
	/tmp/x/main/main.go:6 +0x27
exit status 2

This proposal suggests that the conversion should succeed at run time. I'm asking how that can be implemented.

[Bjohnson131]

@ianlancetaylor Are you asking how to mark it as implementing the interface at compile time? what happens at run-time is irrelevant, that is, unless you can dynamically use reflect to change methods on a type such that they match an interface, and even then, keeping the current handling of the edge cases is OK.

[ianlancetaylor]

For complete clarity, here is the multi-package example on the Go playground: https://play.golang.org/p/TkAA0qzYMAp

[ianlancetaylor]

What happens at run time is not irrelevant. This proposal is saying that the conversion from the type b.S to the type a.I should succeed. That conversion fails today, but under this proposal it should succeed. In this example there is no use of the reflect package.

Do you agree that if we adopt this proposal then the program above should succeed at run time?

If we agree on that, then we need to understand how to implement it. And I do not understand how to do that.

[Bjohnson131]

If there is a run-time check to see if s implements I, then it's easier than explicitly checking all return values. just check that the number of parameters is the same, and that the non interface{} types satisfy the current criterion

[ianlancetaylor]

I agree that it is possible at run time to verify that the conversion can succeed. But that is not enough. We need an actual implementation of M that can be called at run time and that will return a value of type interface{}. The only implementation of M that we have is one that returns string.

[Bjohnson131]

@ianlancetaylor is there any type in go for which _, ok := x.(interface{}); !ok ? that seems like the obvious solution. The other solution is the same one that handles as I mentioned above.

[ianlancetaylor]

is there any type in go for which _, ok := x.(interface{}); !ok ?

No. All types implement the empty interface, by definition.

that seems like the obvious solution.

I'm sorry, I don't understand.

[Bjohnson131]

is there any type in go for which _, ok := x.(interface{}); !ok ?

No. All types implement the empty interface, by definition.

that seems like the obvious solution.

I'm sorry, I don't understand.

well if x == x.(interface{}), then there is no problem...

[ianlancetaylor]

It is true that x == x.(interface{}) will always be true.

But the only way that I can see that that might help is if we always compile every single method to return interface{}. That would be a massive performance hit. We can't do that.

[Bjohnson131]

It is true that x == x.(interface{}) will always be true.

But the only way that I can see that that might help is if we always compile every single method to return interface{}. That would be a massive performance hit. We can't do that.

only the ones that could satisfy interfaces as imported in the code...

[balasanjay]

@Bjohnson131 With Ian's example above, when compiling package b, there are no imported interfaces, so the compiler will not generate M as returning interface{}. It cannot peek into the "future", and know that a user will later write package a such that there is suddenly an interface M returning an interface{}, nor does it know that the user will later use both package a and package b in the same program*.

So the only way for the compiler to safely compile package b under this proposal would be to do as Ian said and compile every method as though it were returning interface{} on the off chance that this becomes necessary. (Which brings us back to a massive performance hit).

---

*: Note that Go uses separate compilation for each package, not a single global compilation for a whole program. At the time that the compiler is compiling package X, it can consider information from X and its immediate dependencies, but not arbitrary other packages from the program that X will be eventually be compiled into.

[mvdan]

It seems to me that this needs to wait for generics to land, at the very least - the two are very related and generics has been in the works for years.

[Bjohnson131]

It seems to me that this needs to wait for generics to land, at the very least - the two are very related and generics has been in the works for years.

True... It seems that Generics is a bigger solution to the problem at hand.

[Bjohnson131]

It cannot peek into the "future", and know that a user will later write package a such that there is suddenly an interface M returning an interface{}

So this would imply that two unreltaed packages can satisfy eachother's interfaces currently without importing them. IIRC this is what the go.mod is for, I've never seen an interface be satisfied from a package that's not in go.mod (or any subsequent go.mod(s)).. Is this not the case?

[ianlancetaylor]

Yes, two unrelated packages can satisfy each other's interfaces without either importing the other. This is separate from go.mod. It's just how the language works. A type with a Read([]byte) (int, error) method implements io.Reader, even if the package defining the type does not import "io".

[ianlancetaylor]

Based on discussion above this is a likely decline. Leaving open for four weeks for final comments.

[andig]

Late to the party. Due to not having generics and a limited set of methods available I'm often doing things like

v.statusG = provider.NewCached(func() (interface{}, error) {
	return v.status()
}, cc.Cache).InterfaceGetter()

just for converting any type to interface.

But the only way that I can see that that might help is if we always compile every single method to return interface{}. That would be a massive performance hit. We can't do that.

One way to do this in the compiler (without compiling everything to return interface) would be to create that stub function at compile time. If that is a worth case of compile change is another question. My example above could definitely be rewritten with generics.

[Bjohnson131]

Yes, two unrelated packages can satisfy each other's interfaces without either importing the other..

But one cannot invoke the Interfacing properties of said structs without at some level importing both packages... I think you're confusing the implementation of type checking with the implementation of interfaces in general... Unless I'm confused, there's no problem that you have presented that isn't already solved by the current compiler, with the exception of the problem of checking interface return types for any other return type, and this is a change that would likely be a low-complexity add.

[balasanjay]

@Bjohnson131 I think its possible you may be confusing the go tool, and the compiler.

The compiler runs once per package, not once per program. For instance, if I have a program where main imports a, and a imports b, and I invoke go build once, the compiler is invoked 3 times. You can see some of these details by running go build -x, which will output the intermediate steps that are being taken.

Consider a program where main depends on a1, a1 depends on a2, a2 depends on a3 and so on until aN.
Similarly, main depends on b1, b1 depends on b2 and so on until bM.

The two unrelated packages we are talking about here could be aN and bM, which are very far from their common parent main. We would not want a Go compiler where each package's compilation depended on the full dependency tree (that way leads to quadratic build time with respect to the build graph, and is how C++ header inclusions work).

[ianlancetaylor]

No change in consensus.