Frequent "failed to get data from low-level cache" warning

[oschwald]

I have been seeing the following warning quite frequently in both our CI environment and on my local development machine:

level=warning msg="[runner/goanalysis_metalinter/goanalysis] Failed to get persisted facts: failed to get data from low-level cache by key printf/facts for package trace: failed to stat file /home/greg/.cache/golangci-lint/af/aff20f6cd29342c0f0dcc2ea1acf50b374a6cf1954a9287664287e2d313d7244-d: stat /home/greg/.cache/golangci-lint/af/aff20f6cd29342c0f0dcc2ea1acf50b374a6cf1954a9287664287e2d313d7244-d: no such file or directory"

There isn't anything else running that should be deleting these files and we aren't running multiple instances of golangci-lint concurrently.

This has been going on for a month or so, likely starting around the time I upgraded to 1.22.0.

Please let me know if I can supply any additional information.

---

Thank you for creating the issue!

• Yes, I'm using a binary release within 2 latest major releases. Only such installations are supported.
• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've included all information below (version, config, etc).

Please include the following information:

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version 1.22.2 built from cb2f8ba on 2019-12-30T19:26:28Z

Config file

$ cat .golangci.tonl
[run]
  deadline = "10m"

  modules-download-mode = "vendor"

  skip-files = [
    "_easyjson\\.go$",
    "_easyjson_test\\.go$",
  ]

  tests = true

[linters]
  disable-all = true
  enable = [
    "deadcode",
    "depguard",
    "errcheck",
    "goconst",
    "gocyclo",
    "gocritic",
    "gofmt",
    "golint",
    "gosec",
    "gosimple",
    "ineffassign",
    "maligned",
    "misspell",
    "nakedret",
    # https://github.com/golangci/golangci-lint/issues/287
    # "safesql",
    "staticcheck",
    "structcheck",
    "typecheck",
    "unconvert",
    "unparam",
    "varcheck",
    "vet",
    "vetshadow",
  ]

[linters-settings.depguard]
  list-type = "blacklist"
  include-go-root = true
  packages = [
    # This library panics frequently on invalid input. Please use
    # github.com/pelletier/go-toml instead.
    "github.com/BurntSushi/toml",

    # The performance of this library is absolutely abysmal. Consider
    # using github.com/avct/uasurfer instead.
    "github.com/ua-parser/uap-go/uaparser",

    # This package is frozen and has a number of issues. We use
    # github.com/RackSec/srslog instead.
    "log/syslog"
  ]

[issues]
exclude-use-default = false

Go environment

$ go version && go env
go version go1.13.5 linux/amd64
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/greg/.cache/go-build"
GOENV="/home/greg/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/greg/MaxMind/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build036139998=/tmp/go-build -gno-record-gcc-switches"

Verbose output of running

$ golangci-lint run -v
INFO [config_reader] Config search paths: [./ /home/greg/MaxMind/go/src/github.maxmind.com/maxmind/mm_website.git /home/greg/MaxMind/go/src/github.maxmind.com/maxmind /home/greg/MaxMind/go/src/github.maxmind.com /home/greg/MaxMind/go/src /home/greg/MaxMind/go /home/greg/MaxMind /home/greg /home /] 
INFO [config_reader] Used config file .golangci.toml 
INFO [lintersdb] Active 21 linters: [deadcode depguard errcheck goconst gocritic gocyclo gofmt golint gosec gosimple govet ineffassign maligned misspell nakedret staticcheck structcheck typecheck unconvert unparam varcheck] 
INFO [lintersdb] Active 21 linters: [deadcode depguard errcheck goconst gocritic gocyclo gofmt golint gosec gosimple govet ineffassign maligned misspell nakedret staticcheck structcheck typecheck unconvert unparam varcheck] 
INFO [loader] Go packages loading at mode 575 (imports|name|types_sizes|compiled_files|deps|exports_file|files) took 2.437594027s 
ERRO Running error: context loading failed: no go files to analyze 
INFO Memory: 26 samples, avg is 68.9MB, max is 68.9MB 
INFO Execution took 2.441272329s                  

[tehsphinx]

Just got this as well and was able to resolve it with golangci-lint cache clean. I'm on Mac with golangci-lint 1.23.7.

[skoval00]

In my case golangci-lint cache clean does not help. I run it on Ubuntu 18.04 inside Windows Subsystem for Linux (WSLv1). Installed it via curl https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ./third_party v1.23.1
Permission is always denied to the same file in cache (see run output below). The only workaround I have found is to run golangci-lint non-concurrently with -j 1. I have a feeling that this is more of a WSL bug, because sometimes I get "permission denied" errors from go build.

Version of golangci-lint

$ third_party/golangci-lint --version
golangci-lint has version 1.23.1 built from 567904e on 2020-01-20T08:00:15Z

Config file

$ cat configs/golangci.yml
# This file contains all available configuration options
# with their default values.

# options for analysis running
run:
  # default concurrency is a available CPU number
  #concurrency: 4

  # timeout for analysis, e.g. 30s, 5m, default is 1m
  deadline: 30m

  # include test files or not, default is true
  tests: true

  # which dirs to skip: they won't be analyzed;
  # can use regexp here: generated.*, regexp is applied on full path;
  # default value is empty list, but next dirs are always skipped independently
  # from this option's value:
  #		vendor$, third_party$, testdata$, examples$, Godeps$, builtin$
  skip-dirs:
    - bin$
    - \.git$

  # which files to skip: they will be analyzed, but issues from them
  # won't be reported. Default value is empty list, but there is
  # no need to include all autogenerated files, we confidently recognize
  # autogenerated files. If it's not please let us know.
#  skip-files:
#    - "_rpc.go"
#    - "_easyjson.go"
#    - ".pb.go"

# all available settings of specific linters
linters-settings:
  errcheck:
    # report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`;
    # default is false: such cases aren't reported by default.
    check-blank: true

  govet:
    # report about shadowed variables
    check-shadowing: true

  golint:
    # minimal confidence for issues, default is 0.8
    min-confidence: 0.3
  gocyclo:
    # minimal code complexity to report, 30 by default (but we recommend 10-20)
    min-complexity: 15
  gocognit:
    # minimal code complexity to report, 30 by default (but we recommend 10-20)
    min-complexity: 20
  dupl:
    # tokens count to trigger issue, 150 by default
    threshold: 200
  lll:
    # max line length, lines longer will be reported. Default is 120.
    # '\t' is counted as 1 character by default, and can be changed with the tab-width option
    line-length: 120
  nakedret:
    # make an issue if func has more lines of code than this setting and it has naked returns; default is 30
    max-func-lines: 30
  funlen:
    lines: 80
    statements: 50
  godox:
    # report any comments starting with keywords, this is useful for TODO or FIXME comments that
    # might be left in the code accidentally and should be resolved before merging
    keywords: # default keywords are TODO, BUG, and FIXME, these can be overwritten by this setting
      - TODO
      - BUG
      - FIXME
      - HACK
  dogsled:
    # checks assignments with too many blank identifiers; default is 2
    max-blank-identifiers: 2
  whitespace:
    multi-if: false   # Enforces newlines (or comments) after every multi-line if statement
    multi-func: false # Enforces newlines (or comments) after every multi-line function signature
  wsl:
    # If true append is only allowed to be cuddled if appending value is
    # matching variables, fields or types on line above. Default is true.
    strict-append: true
    # Allow calls and assignments to be cuddled as long as the lines have any
    # matching variables, fields or types. Default is true.
    allow-assign-and-call: true
    # Allow multiline assignments to be cuddled. Default is true.
    allow-multiline-assign: true
    # Allow declarations (var) to be cuddled.
    allow-cuddle-declarations: false
    # Allow trailing comments in ending of blocks
    allow-trailing-comment: false
    # Force newlines in end of case at this limit (0 = never).
    force-case-trailing-whitespace: 0

linters:
  disable-all: true
  enable:
    - bodyclose
    - deadcode
    - depguard
    - dupl
    - errcheck
    - goconst
    - gocritic
    - gocyclo
    #- godox
    - golint
    - gosec
    - gosimple
    - govet
    - ineffassign
    - interfacer
    - lll
    - misspell
    - nakedret
    - prealloc
    - scopelint
    - staticcheck
    - structcheck
    - stylecheck
    - typecheck
    - unconvert
    - unparam
    - unused
    - varcheck
  #disable:
  #  - goimports        # not needed
  #  - maligned         # not critical for us
  #  - gochecknoglobals # we are using this
  #  - gochecknoinits   # and this
  #  - dogsled          # it is not clear how to fix problems from this linter
  #  - gofmt            # buggy linter and we have script check
  #
  # # new
  #  - gomnd            # supported from 1.22.2
  #  - funlen           # dupl of gocyclo
  #  - gocognit         # the same
  #  - wsl              # imho, useless as linter, needed autoformat tool
  #  - whitespace       # the same
  fast: false

issues:
  max-issues-per-linter: 0
  max-same-issues: 0
  exclude:
    - "should have comment"
    - "always receives"
    - "parameter .* is always"
    - "comment on exported .* should be of the form"
    - "Use of weak cryptographic primitive"
    - "can be `fmt.Stringer`"
    - "can be `github.com/gogo/protobuf/proto.Message`"
    - 'ST1016: methods on the same type should have the same receiver name \(seen \d+x "[^"]+", 4x "v"\)'
    - 'ST1016: methods on the same type should have the same receiver name \(seen 4x "v", \d+x "[^"]+"\)'
    - 'ST1016: methods on the same type should have the same receiver name \(seen \d+x "[^"]+", \d+x "srv"\)'
    - 'ST1016: methods on the same type should have the same receiver name \(seen \d+x "srv", \d+x "[^"]+"\)'

  exclude-rules:
    # Exclude lll issues for long lines with go:generate
    - linters:
        - lll
      source: "^//go:generate "

output:
  format: tab

Go environment

$ go version && go env
go version go1.13.10 linux/amd64
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/skoval/.cache/go-build"
GOENV="/home/skoval/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB="gitlab.some.host.name,stash.some.host.name"
GOOS="linux"
GOPATH="/home/skoval/go"
GOPRIVATE=""
GOPROXY="http://gomods.some.host.name:3000/"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/skoval/src/<some-package>/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build223812328=/tmp/go-build -gno-record-gcc-switches"

Verbose output of running

$ third_party/golangci-lint run -c configs/golangci.yml -j 2 -v ./...
INFO [config_reader] Used config file configs/golangci.yml
INFO [lintersdb] Active 27 linters: [bodyclose deadcode depguard dupl errcheck goconst gocritic gocyclo golint gosec gosimple govet ineffassign interfacer lll misspell nakedret prealloc scopelint
staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck]
INFO [lintersdb] Active 27 linters: [bodyclose deadcode depguard dupl errcheck goconst gocritic gocyclo golint gosec gosimple govet ineffassign interfacer lll misspell nakedret prealloc scopelint
staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck]
INFO [loader] Go packages loading at mode 575 (compiled_files|files|imports|name|types_sizes|deps|exports_file) took 1.8107186s
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 47.3494ms
WARN [runner/goanalysis_metalinter/goanalysis] Failed to get persisted facts: failed to get data from low-level cache by key ctrlflow/facts for package cpu: open /home/skoval/.cache/golangci-lint/e2/e24a307ec1d4f19e6023639f91a2b1e33a9a2bbadcba08e4b8a7765245b19de7-d: permission denied
WARN [runner/goanalysis_metalinter/goanalysis] Failed to get persisted facts: failed to get data from low-level cache by key fact_deprecated/facts for package nettrace: open /home/skoval/.cache/golangci-lint/e2/e24a307ec1d4f19e6023639f91a2b1e33a9a2bbadcba08e4b8a7765245b19de7-d: permission denied
...................
<- repeat many times for different packages though permission is always denied to the same file ->
...................
INFO [runner/goanalysis_metalinter/goanalysis] analyzers took 12.8392508s with top 10 stages: dupl: 1.9391493s, buildssa: 1.9158052s, unconvert: 1.6743365s, golint: 393.8686ms, gosec: 334.6029ms, unparam: 292.9031ms, misspell: 223.4006ms, ineffassign: 219.8455ms, interfacer: 217.6711ms, gocritic: 207.9322ms
INFO [runner/unused/goanalysis] analyzers took 15.171ms with top 10 stages: buildssa: 13.6526ms, U1000: 1.5184ms                                                                                    INFO [runner] Issues before processing: 1088, after processing: 0
INFO [runner] Processors filtering stat (out/in): skip_dirs: 1088/1088, autogenerated_exclude: 526/1088, identifier_marker: 526/526, filename_unadjuster: 1088/1088, path_prettifier: 1088/1088, skip_files: 1088/1088, cgo: 1088/1088, nolint: 0/23, exclude: 23/526, exclude-rules: 23/23
INFO [runner] processing took 196.7461ms with stages: path_prettifier: 117.7133ms, autogenerated_exclude: 41.0978ms, exclude: 18.5811ms, identifier_marker: 8.7294ms, skip_dirs: 4.8355ms, nolint: 4.2475ms, exclude-rules: 1.2188ms, cgo: 223.8µs, filename_unadjuster: 61.9µs, diff: 16.2µs, max_same_issues: 3.5µs, uniq_by_line: 3.3µs, skip_files: 2.9µs, path_shortener: 2.8µs, source_code: 2.8µs, max_from_linter: 2.8µs, max_per_file_from_linter: 2.7µs                                                                                                                                           INFO [runner] linters took 3.6770022s with stages: goanalysis_metalinter: 3.3765984s, unused: 103.3186ms
INFO File cache stats: 124 entries of total size 366.8KiB                                                                                                                                           INFO Memory: 57 samples, avg is 116.6MB, max is 204.8MB
INFO Execution took 5.6052129s

[ernado]

Hey @skoval00 can you please try the latest version?

I think that I/O should be more robust in v1.27

[skoval00]

@ernado I tried v1.27 with the same result. I'll keep running it non-concurrently because I strongly believe this issue may be related to WSL1 file system performance. Right now I can't switch to WSL2 and I am not sure it will be an easy move in my case.

[ernado]

Looks like we should add a hack in robustio/renamio that will detect WSL1 and apply same logic as for windows.

But WSL1 is deprecated in favor of WSL2, so I'm not sure about priority of this issue, sorry :(