Skip to content

Race condition in TestInstanceLimitsUnmarshal #6536

Closed
@dsabsay

Description

@dsabsay

It's happened at least 3 times in the last day:

https://github.com/cortexproject/cortex/actions/runs/12914166852/job/36013114905
https://github.com/cortexproject/cortex/actions/runs/12919554686/job/36030297113?pr=6535

==================
WARNING: DATA RACE
Write at 0x0000048e5c08 by goroutine 52047:
  github.com/cortexproject/cortex/pkg/ingester.TestInstanceLimitsUnmarshal()
      /__w/cortex/cortex/pkg/ingester/instance_limits_test.go:11 +0xc8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1690 +0x226
  testing.(*T).Run.gowrap1()
      /usr/local/go/src/testing/testing.go:1743 +0x44
Previous read at 0x0000048e5c08 by goroutine 51855:
  github.com/cortexproject/cortex/pkg/ingester.(*Ingester).getInstanceLimits()
      /__w/cortex/cortex/pkg/ingester/ingester.go:3171 +0xdc
  github.com/cortexproject/cortex/pkg/ingester.(*Ingester).updateLoop()
      /__w/cortex/cortex/pkg/ingester/ingester.go:927 +0x5a
  github.com/cortexproject/cortex/pkg/ingester.(*Ingester).updateLoop-fm()
      <autogenerated>:1 +0x47
  github.com/cortexproject/cortex/pkg/util/services.(*BasicService).main()
      /__w/cortex/cortex/pkg/util/services/basic_service.go:190 +0x3b7
  github.com/cortexproject/cortex/pkg/util/services.(*BasicService).StartAsync.func1.gowrap1()
      /__w/cortex/cortex/pkg/util/services/basic_service.go:119 +0x33
Goroutine 52047 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:1743 +0x825
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:2168 +0x85
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1690 +0x226
  testing.runTests()
      /usr/local/go/src/testing/testing.go:2166 +0x8be
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:2034 +0xf17
  main.main()
      _testmain.go:221 +0x164
Goroutine 51855 (running) created at:
  github.com/cortexproject/cortex/pkg/util/services.(*BasicService).StartAsync.func1()
      /__w/cortex/cortex/pkg/util/services/basic_service.go:119 +0x1dc
  github.com/cortexproject/cortex/pkg/util/services.(*BasicService).switchState()
      /__w/cortex/cortex/pkg/util/services/basic_service.go:139 +0x115
  github.com/cortexproject/cortex/pkg/util/services.(*BasicService).StartAsync()
      /__w/cortex/cortex/pkg/util/services/basic_service.go:116 +0xb1
  github.com/cortexproject/cortex/pkg/ingester.(*Ingester).StartAsync()
      <autogenerated>:1 +0x5b
  github.com/cortexproject/cortex/pkg/util/services.StartAndAwaitRunning()
      /__w/cortex/cortex/pkg/util/services/services.go:104 +0x49
  github.com/cortexproject/cortex/pkg/ingester.TestIngester_UpdateLabelSetMetrics()
      /__w/cortex/cortex/pkg/ingester/ingester_test.go:6378 +0xa47
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1690 +0x226
  testing.(*T).Run.gowrap1()
      /usr/local/go/src/testing/testing.go:1743 +0x44
==================
--- FAIL: TestInstanceLimitsUnmarshal (0.01s)
    testing.go:1399: race detected during execution of test
FAIL
FAIL	github.com/cortexproject/cortex/pkg/ingester	231.535s

This test is mutating a package global:

defaultInstanceLimits = &InstanceLimits{

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions