Faster rolling upgrades. #325
Conversation
tomwilkie
force-pushed
the
321-faster-rolling-upgrade
branch
from
f06ac7b to
2c1a25c
Compare
cortex.proto
Outdated
There was a problem hiding this comment.
Rename to Transfer{StartStreamFinish}?
Also, can the {Start,Finish} methods be avoided?
ingester/ingester_claim.go
Outdated
There was a problem hiding this comment.
Move to util/compat.go
There was a problem hiding this comment.
Nah, uses the ingester/desc type which I want to keep internal.
tomwilkie
force-pushed
the
321-faster-rolling-upgrade
branch
from
4f81de5 to
587f34a
Compare
cortex.proto
Outdated
There was a problem hiding this comment.
Say which of the two is supposed to call this.
distributor/distributor.go
Outdated
There was a problem hiding this comment.
Please update comment say what string and time.Duration represent
ingester/ingester.go
Outdated
ingester/ingester.go
Outdated
ingester/ingester.go
Outdated
There was a problem hiding this comment.
Its shared with the server config - see https://github.com/weaveworks/cortex/blob/master/cmd/ingester/main.go#L33
ring/model.go
Outdated
There was a problem hiding this comment.
"returning the claimed tokens"
ring/model.go
Outdated
ingester/util.go
Outdated
There was a problem hiding this comment.
Please remove the implementation that's in alertmanager.
There was a problem hiding this comment.
They do different things - alertmangers gets the first non-empty address, this gets the address for a given interface. I'll file a ticket to reconcile them later.
ingester/ingester_lifecycle_test.go
Outdated
There was a problem hiding this comment.
Can we use testify, so we can spelling this require.NoError(err)?
There was a problem hiding this comment.
Thanks! If you're really keen, you can also import assert from testify and use its Equal function to replace some of the DeepEqual business.
ingester/ingester_lifecycle_test.go
Outdated
There was a problem hiding this comment.
What about restart are we testing? (Please add comment)
tomwilkie
force-pushed
the
321-faster-rolling-upgrade
branch
from
8113f83 to
b14c4a5
Compare
ingester/ingester_claim.go
Outdated
|
|
||
| // We can't send "extra" fields with a streaming call, so we repeat | ||
| // wireSeries.FromIngesterId and assume it is the same every time | ||
| // tound this loop. |
ingester/ingester_lifecycle.go
Outdated
There was a problem hiding this comment.
FWIW, "by" or "from" are probably better than the "off of the", but it's such an incredibly minor point.
|
I missed this. Sorry, I don't have a better suggestion. No strong
objections to `c` but don't think it's a huge improvement either. No big
deal.
…On Mon, 13 Mar 2017 at 11:34 Tom Wilkie ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In ingester/ingester_lifecycle.go
<#325 (comment)>:
> + }
+
+ ringDesc, err := i.consul.Get(ring.ConsulKey)
+ if err != nil {
+ log.Error("Error talking to consul: %v", err)
+ return false
+ }
+
+ i.ready = i.ready || ringDesc.(*ring.Desc).Ready(i.cfg.ringConfig.HeartbeatTimeout)
+ return i.ready
+}
+
+// ChangeState of the ingester, for use off the loop() goroutine.
+func (i *Ingester) ChangeState(state ring.IngesterState) error {
+ err := make(chan error)
+ i.actorChan <- func() {
better suggestion? c?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#325 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAHq6kpNP4rQyYFxmDpvUgURwNKKvlyGks5rlSmzgaJpZM4MYGwP>
.
|
Fixes #321
It works! And its quick...
TODO
ensure ingester flushing stops before transferring chunks- shouldn't do this, as we don't want chunks that won't flush to block the transfer.add an ever-extending timeout on the receiving ingester, when we do timeout auto-join- going to leave this for now, can always kill ingesters if they get stuck, will fall back to flushing.