Skip to content

Commit 537b34c

Browse files
tomwilkiepracucci
tomwilkie
and
pracucci
authored
Add /config handler which exposes the current YAML config of cortex. (#2165)
* Add /config handler which exposes the current YAML config of cortex. Signed-off-by: Tom Wilkie <[email protected]> Signed-off-by: Tom Wilkie <[email protected]> * Add changelog and line to docs. Signed-off-by: Tom Wilkie <[email protected]> Signed-off-by: Tom Wilkie <[email protected]> * Fix lint. (Also regen docs) Signed-off-by: Tom Wilkie <[email protected]> Signed-off-by: Tom Wilkie <[email protected]> * Fix marshalling / unmarshalling of flagext values and add test. Signed-off-by: Tom Wilkie <[email protected]> Signed-off-by: Tom Wilkie <[email protected]> * Add PR number to changelog. Signed-off-by: Tom Wilkie <[email protected]> Signed-off-by: Tom Wilkie <[email protected]> * Fixed linter Signed-off-by: Marco Pracucci <[email protected]> * Fixed URLValue zero value marshalling Signed-off-by: Marco Pracucci <[email protected]> * Removed omitempty from LifecyclerConfig Signed-off-by: Marco Pracucci <[email protected]> * Added integration test on GET /config endpoint Signed-off-by: Marco Pracucci <[email protected]> Co-authored-by: Marco Pracucci <[email protected]>
1 parent b00cb5f commit 537b34c

File tree

10 files changed

+236
-8
lines changed

10 files changed

+236
-8
lines changed

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
* [FEATURE] Added sharding support to compactor when using the experimental TSDB blocks storage. #2113
3131
* [FEATURE] Add ability to override YAML config file settings using environment variables. #2147
3232
* `-config.expand-env`
33+
* [FEATURE] Add /config HTTP endpoint which exposes the current Cortex configuration as YAML. #2165
3334
* [ENHANCEMENT] Add `status` label to `cortex_alertmanager_configs` metric to gauge the number of valid and invalid configs. #2125
3435
* [ENHANCEMENT] Cassandra Authentication: added the `custom_authenticators` config option that allows users to authenticate with cassandra clusters using password authenticators that are not approved by default in [gocql](https://github.com/gocql/gocql/blob/81b8263d9fe526782a588ef94d3fa5c6148e5d67/conn.go#L27) #2093
3536
* [ENHANCEMENT] Experimental TSDB: Export TSDB Syncer metrics from Compactor component, they are prefixed with `cortex_compactor_`. #2023

‎docs/configuration/config-file-reference.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ slug: configuration-file
77

88
Cortex can be configured using a YAML file - specified using the `-config.file` flag - or CLI flags. In case you combine both, CLI flags take precedence over the YAML config file.
99

10+
The current configuration of any Cortex component can be seen by visiting the `/config` HTTP path.
11+
1012
## Reference
1113

1214
To specify which configuration file to load, pass the `-config.file` flag at the command line. The file is written in [YAML format](https://en.wikipedia.org/wiki/YAML), defined by the scheme below. Brackets indicate that a parameter is optional.

‎docs/configuration/config-file-reference.template‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ slug: configuration-file
77

88
Cortex can be configured using a YAML file - specified using the `-config.file` flag - or CLI flags. In case you combine both, CLI flags take precedence over the YAML config file.
99

10+
The current configuration of any Cortex component can be seen by visiting the `/config` HTTP path.
11+
1012
## Reference
1113

1214
To specify which configuration file to load, pass the `-config.file` flag at the command line. The file is written in [YAML format](https://en.wikipedia.org/wiki/YAML), defined by the scheme below. Brackets indicate that a parameter is optional.

‎integration/api_config_test.go‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
package main
2+
3+
import (
4+
"fmt"
5+
"io/ioutil"
6+
"net/http"
7+
"path/filepath"
8+
"testing"
9+
10+
"github.com/stretchr/testify/require"
11+
"github.com/thanos-io/thanos/pkg/runutil"
12+
13+
"github.com/cortexproject/cortex/integration/e2e"
14+
"github.com/cortexproject/cortex/integration/e2ecortex"
15+
)
16+
17+
func TestConfigAPIEndpoint(t *testing.T) {
18+
s, err := e2e.NewScenario(networkName)
19+
require.NoError(t, err)
20+
defer s.Close()
21+
22+
// Start Cortex in single binary mode, reading the config from file.
23+
require.NoError(t, copyFileToSharedDir(s, "docs/configuration/single-process-config.yaml", cortexConfigFile))
24+
flags := map[string]string{
25+
"-config.file": filepath.Join(e2e.ContainerSharedDir, cortexConfigFile),
26+
}
27+
28+
cortex1 := e2ecortex.NewSingleBinary("cortex-1", flags, "", 9009)
29+
require.NoError(t, s.StartAndWaitReady(cortex1))
30+
31+
// Get config from /config API endpoint.
32+
res, err := e2e.GetRequest(fmt.Sprintf("http://%s/config", cortex1.Endpoint(9009)))
33+
require.NoError(t, err)
34+
35+
defer runutil.ExhaustCloseWithErrCapture(&err, res.Body, "config API response")
36+
body, err := ioutil.ReadAll(res.Body)
37+
require.NoError(t, err)
38+
require.Equal(t, http.StatusOK, res.StatusCode)
39+
40+
// Start again Cortex in single binary with the exported config
41+
// and ensure it starts (pass the readiness probe).
42+
require.NoError(t, writeFileToSharedDir(s, cortexConfigFile, body))
43+
cortex2 := e2ecortex.NewSingleBinary("cortex-2", flags, "", 9009)
44+
require.NoError(t, s.StartAndWaitReady(cortex2))
45+
}

‎pkg/cortex/modules.go‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ import (
1818
"github.com/weaveworks/common/middleware"
1919
"github.com/weaveworks/common/server"
2020
"google.golang.org/grpc/health/grpc_health_v1"
21+
"gopkg.in/yaml.v2"
2122

2223
"github.com/cortexproject/cortex/pkg/alertmanager"
2324
"github.com/cortexproject/cortex/pkg/chunk"
@@ -167,9 +168,28 @@ func (m *moduleName) UnmarshalYAML(unmarshal func(interface{}) error) error {
167168
return m.Set(s)
168169
}
169170

170-
func (t *Cortex) initServer(cfg *Config) (err error) {
171+
func (t *Cortex) initServer(cfg *Config) error {
172+
var err error
171173
t.server, err = server.New(cfg.Server)
172-
return
174+
if err != nil {
175+
return err
176+
}
177+
178+
t.server.HTTP.HandleFunc("/config", func(w http.ResponseWriter, _ *http.Request) {
179+
out, err := yaml.Marshal(cfg)
180+
if err != nil {
181+
http.Error(w, err.Error(), http.StatusInternalServerError)
182+
return
183+
}
184+
185+
w.Header().Set("Content-Type", "text/yaml")
186+
w.WriteHeader(http.StatusOK)
187+
if _, err := w.Write(out); err != nil {
188+
level.Error(util.Logger).Log("msg", "error writing response", "err", err)
189+
}
190+
})
191+
192+
return nil
173193
}
174194

175195
func (t *Cortex) stopServer() (err error) {

‎pkg/ring/lifecycler.go‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,14 +44,14 @@ type LifecyclerConfig struct {
4444

4545
// Config for the ingester lifecycle control
4646
ListenPort *int `yaml:"-"`
47-
NumTokens int `yaml:"num_tokens,omitempty"`
48-
HeartbeatPeriod time.Duration `yaml:"heartbeat_period,omitempty"`
49-
ObservePeriod time.Duration `yaml:"observe_period,omitempty"`
50-
JoinAfter time.Duration `yaml:"join_after,omitempty"`
51-
MinReadyDuration time.Duration `yaml:"min_ready_duration,omitempty"`
47+
NumTokens int `yaml:"num_tokens"`
48+
HeartbeatPeriod time.Duration `yaml:"heartbeat_period"`
49+
ObservePeriod time.Duration `yaml:"observe_period"`
50+
JoinAfter time.Duration `yaml:"join_after"`
51+
MinReadyDuration time.Duration `yaml:"min_ready_duration"`
5252
InfNames []string `yaml:"interface_names"`
5353
FinalSleep time.Duration `yaml:"final_sleep"`
54-
TokensFilePath string `yaml:"tokens_file_path,omitempty"`
54+
TokensFilePath string `yaml:"tokens_file_path"`
5555

5656
// For testing, you can override the address and ID of this ingester
5757
Addr string `yaml:"address" doc:"hidden"`

‎pkg/util/flagext/day.go‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,17 @@ func (v *DayValue) Set(s string) error {
4343
func (v *DayValue) IsSet() bool {
4444
return v.set
4545
}
46+
47+
// UnmarshalYAML implements yaml.Unmarshaler.
48+
func (v *DayValue) UnmarshalYAML(unmarshal func(interface{}) error) error {
49+
var s string
50+
if err := unmarshal(&s); err != nil {
51+
return err
52+
}
53+
return v.Set(s)
54+
}
55+
56+
// MarshalYAML implements yaml.Marshaler.
57+
func (v DayValue) MarshalYAML() (interface{}, error) {
58+
return v.Time.Time().Format("2006-01-02"), nil
59+
}

‎pkg/util/flagext/day_test.go‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
package flagext
2+
3+
import (
4+
"testing"
5+
6+
"github.com/stretchr/testify/assert"
7+
"github.com/stretchr/testify/require"
8+
"gopkg.in/yaml.v2"
9+
)
10+
11+
func TestDayValueYAML(t *testing.T) {
12+
// Test embedding of DayValue.
13+
{
14+
type TestStruct struct {
15+
Day DayValue `yaml:"day"`
16+
}
17+
18+
var testStruct TestStruct
19+
require.NoError(t, testStruct.Day.Set("1985-06-02"))
20+
expected := []byte(`day: "1985-06-02"
21+
`)
22+
23+
actual, err := yaml.Marshal(testStruct)
24+
require.NoError(t, err)
25+
assert.Equal(t, expected, actual)
26+
27+
var actualStruct TestStruct
28+
err = yaml.Unmarshal(expected, &actualStruct)
29+
require.NoError(t, err)
30+
assert.Equal(t, testStruct, actualStruct)
31+
}
32+
33+
// Test pointers of DayValue.
34+
{
35+
type TestStruct struct {
36+
Day *DayValue `yaml:"day"`
37+
}
38+
39+
var testStruct TestStruct
40+
testStruct.Day = &DayValue{}
41+
require.NoError(t, testStruct.Day.Set("1985-06-02"))
42+
expected := []byte(`day: "1985-06-02"
43+
`)
44+
45+
actual, err := yaml.Marshal(testStruct)
46+
require.NoError(t, err)
47+
assert.Equal(t, expected, actual)
48+
49+
var actualStruct TestStruct
50+
err = yaml.Unmarshal(expected, &actualStruct)
51+
require.NoError(t, err)
52+
assert.Equal(t, testStruct, actualStruct)
53+
}
54+
}

‎pkg/util/flagext/url.go‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,21 @@ func (v *URLValue) UnmarshalYAML(unmarshal func(interface{}) error) error {
3131
if err := unmarshal(&s); err != nil {
3232
return err
3333
}
34+
35+
// An empty string means no URL has been configured.
36+
if s == "" {
37+
v.URL = nil
38+
return nil
39+
}
40+
3441
return v.Set(s)
3542
}
43+
44+
// MarshalYAML implements yaml.Marshaler.
45+
func (v URLValue) MarshalYAML() (interface{}, error) {
46+
if v.URL == nil {
47+
return "", nil
48+
}
49+
50+
return v.URL.String(), nil
51+
}

‎pkg/util/flagext/url_test.go‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
package flagext
2+
3+
import (
4+
"testing"
5+
6+
"github.com/stretchr/testify/assert"
7+
"github.com/stretchr/testify/require"
8+
"gopkg.in/yaml.v2"
9+
)
10+
11+
func TestURLValueYAML(t *testing.T) {
12+
// Test embedding of URLValue.
13+
{
14+
type TestStruct struct {
15+
URL URLValue `yaml:"url"`
16+
}
17+
18+
var testStruct TestStruct
19+
require.NoError(t, testStruct.URL.Set("http://google.com"))
20+
expected := []byte(`url: http://google.com
21+
`)
22+
23+
actual, err := yaml.Marshal(testStruct)
24+
require.NoError(t, err)
25+
assert.Equal(t, expected, actual)
26+
27+
var actualStruct TestStruct
28+
err = yaml.Unmarshal(expected, &actualStruct)
29+
require.NoError(t, err)
30+
assert.Equal(t, testStruct, actualStruct)
31+
}
32+
33+
// Test pointers of URLValue.
34+
{
35+
type TestStruct struct {
36+
URL *URLValue `yaml:"url"`
37+
}
38+
39+
var testStruct TestStruct
40+
testStruct.URL = &URLValue{}
41+
require.NoError(t, testStruct.URL.Set("http://google.com"))
42+
expected := []byte(`url: http://google.com
43+
`)
44+
45+
actual, err := yaml.Marshal(testStruct)
46+
require.NoError(t, err)
47+
assert.Equal(t, expected, actual)
48+
49+
var actualStruct TestStruct
50+
err = yaml.Unmarshal(expected, &actualStruct)
51+
require.NoError(t, err)
52+
assert.Equal(t, testStruct, actualStruct)
53+
}
54+
55+
// Test no url set in URLValue.
56+
{
57+
type TestStruct struct {
58+
URL URLValue `yaml:"url"`
59+
}
60+
61+
var testStruct TestStruct
62+
expected := []byte(`url: ""
63+
`)
64+
65+
actual, err := yaml.Marshal(testStruct)
66+
require.NoError(t, err)
67+
assert.Equal(t, expected, actual)
68+
69+
var actualStruct TestStruct
70+
err = yaml.Unmarshal(expected, &actualStruct)
71+
require.NoError(t, err)
72+
assert.Equal(t, testStruct, actualStruct)
73+
}
74+
}

0 commit comments

Comments
 (0)