Fix ingesters with less tokens stuck in LEAVING#5061

alanprot merged 4 commits intocortexproject:mastercortexproject/cortex:masterfrom

marianafranco:fix-stuck-in-leavingmarianafranco/cortex:fix-stuck-in-leavingCopy head branch name to clipboard