Change log level for 4xx on Push API. Do not log on 429 (#5103)

* Remove error logs in case API returns 4xx

Signed-off-by: Daniel Deluiggi <[email protected]>

* Update chnagelog

Signed-off-by: Daniel Deluiggi <[email protected]>

* Change to just not log in 429

Signed-off-by: Daniel Deluiggi <[email protected]>

Signed-off-by: Daniel Deluiggi <[email protected]>

Author: danielblando

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## master / unreleased
 * [CHANGE] Alertmanager: Local file disclosure vulnerability in OpsGenie configuration has been fixed. #5045
+* [CHANGE] Distributor/Ingester: Log warn level on push requests when they have status code 4xx. Do not log if status is 429. #5103
 * [ENHANCEMENT] Update Go version to 1.19.3. #4988
 * [ENHANCEMENT] Querier: limit series query to only ingesters if `start` param is not specified. #4976
 * [ENHANCEMENT] Query-frontend/scheduler: add a new limit `frontend.max-outstanding-requests-per-tenant` for configuring queue size per tenant. Started deprecating two flags `-query-scheduler.max-outstanding-requests-per-tenant` and `-querier.max-outstanding-requests-per-tenant`, and change their value default to 0. Now if both the old flag and new flag are specified, the old flag's queue size will be picked. #5005

pkg/util/push/push.go

@@ -47,8 +47,10 @@ func Handler(maxRecvMsgSize int, sourceIPs *middleware.SourceIPExtractor, push F
 				http.Error(w, err.Error(), http.StatusInternalServerError)
 				return
 			}
-			if resp.GetCode() != 202 {
+			if resp.GetCode()/100 == 5 {
 				level.Error(logger).Log("msg", "push error", "err", err)
+			} else if resp.GetCode() != http.StatusAccepted && resp.GetCode() != http.StatusTooManyRequests {
+				level.Warn(logger).Log("msg", "push refused", "err", err)
 			}
 			http.Error(w, string(resp.Body), int(resp.Code))
 		}