spec: define initialization order more precisely

[the-zucc]

Context

Package Initialization

When importing a package, it is initialized according to the sequence described in the Go specification. If that package has dependencies, it makes sense that these are initialized first, and that is what is described in the spec. In many instances, this is very useful and helps to write very succinct code, eliminating much boilerplate code.

Importing for Side Effects

In the Go specification, there is mention that it is possible to import packages for their side-effects:

An import declaration declares a dependency relation between the importing and imported package. It is illegal for a package to import itself, directly or indirectly, or to directly import a package without referring to any of its exported identifiers. To import a package solely for its side-effects (initialization), use the blank identifier as explicit package name:

import _ "lib/math"

Small Issue

Currently, the import order is predictable for packages with import relations. This is very useful and works flawlessly. However, as I've come to realize it, within a series of import statements in a single file, a package that is imported (for its side effects) early in the list of imports has no guarantee of early initialization. Consequently, the initializations are not ran in order, which makes results unpredictable at times.

Example Use Case - Mocking An Execution Environment Before Running Unit Tests

The package names here are not necessarily representative of what has caused this issue, they are just to explain the use case; let's say I have a package, packagea, that initializes a connection to a server, either in its init() or via variable initializers.

package somepackage_test

import _ "github.com/the-zucc/packageb" // this init() function sets an environment variable containing the server URL
import "github.com/the-zucc/packagea" // this package's init() function reads the environment variable

// Just imagine some unit tests here... :)

In this case, I've observed 2 things:

• There's no guarantee that packageb will be imported and initialized before packagea -- in my case, the package imported the earliest was initialized at the end, effectively not propagating its side effects early enough in the program execution for them to apply to other package initializations, as the environment variable was not set which caused packagea to default to the "production code" server URL, and not the testing URL;
• There isn't anything in the documentation or in the official Go specification referring to package initialization order within a series of import statements, or within a grouped import.

Proposed Solution

Enable some way of specifying that a package should be imported before any other package for its side effects. As an example, initialization behavior for grouped imports could remain the same, but for separate import statements, statement order could define the initialization order.

import _ "github.com/the-zucc/packageb" // this could be initialized first

import ( // and then the initialization behaviour could remain the same for this grouped import
    "github.com/the-zucc/packagec"
    "github.com/the-zucc/packagea"
)

EDIT: as mentioned in the comments, the proposed solution wouldn't work, unfortunately; however it still gives an idea of what a candidate implementation could look like.

[ianlancetaylor]

CC @griesemer

[seankhliao]

Given the widespread use of goimports and similar tools, the proposed solution won't work as it would be rewritten into a single block with sorted ordering.

More generally, what if a package does:

import _ "github.com/the-zucc/packageb" 

import (
    "github.com/the-zucc/packagec"
    "github.com/the-zucc/packagea"
)

while another does

import _ "github.com/the-zucc/packagec"

import (
    "github.com/the-zucc/packageb"
    "github.com/the-zucc/packagea"
)

or even different files in the same package?

[seankhliao]

see also discussions in the other direction, pointing to perhaps the more common need to protect against people writing init functions

removing init #43731
control over defering init #48174
import without running init #38450

[the-zucc]

Given the widespread use of goimports and similar tools, the proposed solution won't work as it would be rewritten into a single block with sorted ordering.

Depending on the implementation of this, goimports could need an update anyway.

More generally, what if a package does:

import _ "github.com/the-zucc/packageb" 

import (
    "github.com/the-zucc/packagec"
    "github.com/the-zucc/packagea"
)

while another does

import _ "github.com/the-zucc/packagec"

import (
    "github.com/the-zucc/packageb"
    "github.com/the-zucc/packagea"
)

or even different files in the same package?

That is a great question... couldn't this raise a syntax error ? Like a "pick a lane bud, you're drunk !" 😄

[seankhliao]

packaged does the first one, packagee does the second
you import both packaged and packagee, but not any of packagea, packageb, packagec directly

are you just never allowed to import these two packages together?

[the-zucc]

packaged does the first one, packagee does the second you import both packaged and packagee, but not any of packagea, packageb, packagec directly

are you just never allowed to import these two packages together?

Without having thought about it too much (therefore don't take my word for it), I think I would simply raise a syntax error on both import lines (for packaged and packagee...) There is hardly any reason why one would want to have 2 different import orders, and if there was a reason, that reason doesn't exist as of today, because at the moment, package initialization order isn't guaranteed (and so we can eliminate that problem by raising a syntax error.)

[the-zucc]

There is hardly any reason why one would want to have 2 different import orders, and if there was a reason, that reason doesn't exist as of today, because at the moment, package initialization order isn't guaranteed (and so we can eliminate that problem by raising a syntax error.)

Actually, I see a potential problem with applications with huge dependency trees with open source packages. One would want to take this into consideration if something is to be done...

[rittneje]

You should not connect to a server from init() as that violates the concept of writing testable code, as you have encountered.

Instead, packagea should have a proper constructor that gets invoked. Each unit test is then responsible for calling it. This establishes proper isolation between each test case, which is a very useful property.

Meanwhile, unit tests for other packages that depend on packagea should likely be using some interface instead of direct function calls, which allows for mocking.

[griesemer]

See also #31636. You can force the init order by ordering the imports accordingly and into "blocks" so that they don't get re-ordered (example). Note that this is not prescribed by the language, it's an implementation behavior.

That said, it is much more robust to be explicit and call initialization functions explicitly if they cause side effects.

[the-zucc]

You should not connect to a server from init() as that violates the concept of writing testable code, as you have encountered.

I guess you are right, there are probably other ways to get succint code without connecting to a server in the init() function. However the fundamental question regarding side effects is still of interest, for other use cases.