diff --git a/content/nginx/admin-guide/load-balancer/http-load-balancer.md b/content/nginx/admin-guide/load-balancer/http-load-balancer.md index a01b69aed..c1a17503f 100644 --- a/content/nginx/admin-guide/load-balancer/http-load-balancer.md +++ b/content/nginx/admin-guide/load-balancer/http-load-balancer.md @@ -14,17 +14,17 @@ type: Load balancing across multiple application instances is a commonly used technique for optimizing resource utilization, maximizing throughput, reducing latency, and ensuring fault‑tolerant configurations. -Watch the [F5 NGINX Plus for Load Balancing and Scaling](https://www.nginx.com/resources/webinars/nginx-plus-for-load-balancing-30-min/) webinar on demand for a deep dive on techniques that NGINX users employ to build large‑scale, highly available web services. +The [F5 NGINX Plus for Load Balancing and Scaling](https://www.nginx.com/resources/webinars/nginx-plus-for-load-balancing-30-min/) webinar offers a deep dive on techniques to build large‑scale, highly available web services. NGINX and NGINX Plus can be used in different deployment scenarios as a [very efficient HTTP load balancer](https://www.nginx.com/blog/nginx-load-balance-deployment-models/). -## Proxying HTTP Traffic to a Group of Servers +## Proxy HTTP Traffic to a Group of Servers -To start using NGINX Plus or NGINX Open Source to load balance HTTP traffic to a group of servers, first you need to define the group with the [`upstream`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) directive. The directive is placed in the [`http`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context. +The first step in using NGINX Plus or NGINX Open Source to load balance across a group of servers is to define the group of servers. Define the group of servers with the [`upstream`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) directive. The directive is placed in the [`http`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context. -Servers in the group are configured using the [`server`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server) directive (not to be confused with the `server` block that defines a virtual server running on NGINX). For example, the following configuration defines a group named **backend** and consists of three server configurations (which may resolve in more than three actual servers): +Servers in the group are configured using the [`server`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server) directive (not to be confused with the `server` block that defines a virtual server running on NGINX). For example, the following configuration defines a group named **backend** and consists of three server configurations. (This may resolve to more than three actual servers): ```nginx http { @@ -36,7 +36,9 @@ http { } ``` -To pass requests to a server group, the name of the group is specified in the [`proxy_pass`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive (or the [`fastcgi_pass`](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass), [`memcached_pass`](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_pass), [`scgi_pass`](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass), or [`uwsgi_pass`](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass) directives for those protocols.) In the next example, a virtual server running on NGINX passes all requests to the **backend** upstream group defined in the previous example: +To pass requests to a server group, the name of the group is specified in the [`proxy_pass`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive. If using an alternate protocol such as fastcgi, use that protocol's pass directive instead. The list of alternate pass directives is: [`fastcgi_pass`](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass), [`memcached_pass`](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_pass), [`scgi_pass`](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass), [`uwsgi_pass`](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass). + +In the next example, a virtual server running on NGINX passes all requests to the **backend** upstream group defined in the previous example: ```nginx server { @@ -46,7 +48,7 @@ server { } ``` -The following example combines the two snippets above and shows how to proxy HTTP requests to the **backend** server group. The group consists of three servers, two of them running instances of the same application while the third is a backup server. Because no load‑balancing algorithm is specified in the `upstream` block, NGINX uses the default algorithm, Round Robin: +The following example combines the two snippets above and shows how to proxy HTTP requests to the **backend** server group. The group consists of three servers, two of them running instances of the same application while the third is a backup server. ```nginx http { @@ -66,11 +68,14 @@ http { -## Choosing a Load-Balancing Method +## Choosing a Load Balancing Method + +NGINX Open Source supports four load balancing methods: Round Robin, Least Connections, IP Hash, and Generic Hash. +NGINX Plus supports six load balancing methods: the four above, Least Time, and Random. -NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds two more methods: +> **Note:** When configuring any method other than Round Robin, put the corresponding directive ([`hash`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash), [`ip_hash`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash), [`least_conn`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn), [`least_time`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time), or [`random`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random)) above the list of `server` directives in the [`upstream {}`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) block. -1. Round Robin – Requests are distributed evenly across the servers, with [server weights](#weights) taken into consideration. This method is used by default (there is no directive for enabling it): +1. Round Robin – Requests are distributed evenly across the servers, with [server weights](#weights) taken into consideration. This method is used by default; there is no directive for enabling it. ```nginx upstream backend { @@ -80,7 +85,7 @@ NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds tw } ``` -2. [Least Connections](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn) – A request is sent to the server with the least number of active connections, again with [server weights](#weights) taken into consideration: +2. [Least Connections](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn) – A request is sent to the server with the least number of active connections. This method also takes [server weights](#weights) into consideration. ```nginx upstream backend { @@ -90,7 +95,7 @@ NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds tw } ``` -3. [IP Hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash) – The server to which a request is sent is determined from the client IP address. In this case, either the first three octets of the IPv4 address or the whole IPv6 address are used to calculate the hash value. The method guarantees that requests from the same address get to the same server unless it is not available. +3. [IP Hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash) – The server to which a request is sent is determined from the client IP address. In this case, either the first three octets of the IPv4 address or the whole IPv6 address is used to calculate the hash value. The method guarantees that requests from the same address get to the same server unless it is not available. ```nginx upstream backend { @@ -100,7 +105,7 @@ NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds tw } ``` - If one of the servers needs to be temporarily removed from the load‑balancing rotation, it can be marked with the [down](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#down) parameter in order to preserve the current hashing of client IP addresses. Requests that were to be processed by this server are automatically sent to the next server in the group: + If one of the servers needs to be temporarily removed from the load‑balancing rotation, it can be marked with the [down](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#down) parameter. This preserves the current hashing of client IP addresses. Requests that were to be processed by this server are automatically sent to the next server in the group. ```nginx upstream backend { @@ -110,7 +115,7 @@ NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds tw } ``` -4. Generic [Hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash) – The server to which a request is sent is determined from a user‑defined key which can be a text string, variable, or a combination. For example, the key may be a paired source IP address and port, or a URI as in this example: +4. Generic [Hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash) – The server to which a request is sent is determined from a user‑defined key. This key can be a text string, a variable, or a combination. For example, the key may be a paired source IP address and port. This example uses a URI: ```nginx upstream backend { @@ -120,9 +125,9 @@ NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds tw } ``` - The optional [consistent](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash) parameter to the `hash` directive enables [ketama](http://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-_a_consistent_hashing_algo_for_memcache_clients) consistent‑hash load balancing. Requests are evenly distributed across all upstream servers based on the user‑defined hashed key value. If an upstream server is added to or removed from an upstream group, only a few keys are remapped which minimizes cache misses in the case of load‑balancing cache servers or other applications that accumulate state. + The optional [consistent](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash) parameter to the `hash` directive enables [ketama](http://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-_a_consistent_hashing_algo_for_memcache_clients) consistent‑hash load balancing. Requests are evenly distributed across all upstream servers based on the user‑defined hashed key value. If an upstream server is added to or removed from an upstream group, only a few keys are remapped, which minimizes cache misses. This is useful for load balancing cache servers or other applications that accumulate state. -5. [Least Time](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time) (NGINX Plus only) – For each request, NGINX Plus selects the server with the lowest average latency and the lowest number of active connections, where the lowest average latency is calculated based on which of the following [parameters](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time) to the `least_time` directive is included: +5. [Least Time](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time) (NGINX Plus only) – For each request, NGINX Plus selects the server with the lowest average latency and the lowest number of active connections. The lowest average latency is calculated based the [parameter](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time) included with the `least_time` directive. This parameter can be one of the following: - `header` – Time to receive the first byte from the server - `last_byte` – Time to receive the full response from the server @@ -136,8 +141,8 @@ NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds tw } ``` -6. [Random](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random) – Each request will be passed to a randomly selected server. -If the `two` parameter is specified, first, NGINX randomly selects two servers taking into account server weights, and then chooses one of these servers using the specified method: +6. [Random](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random) (NGINX Plus only) – Each request will be passed to a randomly selected server. This method takes into account server weights. +If the `two` parameter is specified, NGINX first randomly selects two servers, then chooses between these servers using one of the following specified methods: - `least_conn` – The least number of active connections - `least_time=header` (NGINX Plus) – The least average time to receive the response header from the server ([`$upstream_header_time`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_header_time)) @@ -153,15 +158,14 @@ If the `two` parameter is specified, first, NGINX randomly selects two servers t } ``` - The **Random** load balancing method should be used for distributed environments where multiple load balancers are passing requests to the same set of backends. For environments where the load balancer has a full view of all requests, use other load balancing methods, such as round robin, least connections and least time. + The **Random** load balancing method should be used for distributed environments where multiple load balancers are passing requests to the same set of backends. For environments where the load balancer has a full view of all requests, use other load balancing methods. -> **Note:** When configuring any method other than Round Robin, put the corresponding directive ([`hash`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash), [`ip_hash`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash), [`least_conn`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn), [`least_time`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time), or [`random`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random)) above the list of `server` directives in the [`upstream {}`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) block. ## Server Weights -By default, NGINX distributes requests among the servers in the group according to their weights using the Round Robin method. The [`weight`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#weight) parameter to the [`server`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server) directive sets the weight of a server; the default is `1`: +Some load balancing methods, including Round Robin, Least Connections, and Random, distribute requests according to their server weights. The [`weight`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#weight) parameter to the [`server`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server) directive sets the weight of a server. If no weight is set, it defaults to `1`: ```nginx upstream backend { @@ -171,15 +175,15 @@ upstream backend { } ``` -In the example, **backend1.example.com** has weight `5`; the other two servers have the default weight (`1`), but the one with IP address `192.0.0.1` is marked as a `backup` server and does not receive requests unless both of the other servers are unavailable. With this configuration of weights, out of every `6` requests, `5` are sent to **backend1.example.com** and `1` to **backend2.example.com**. +In the example above, **backend1.example.com** has weight `5`, while the other two servers have the default weight (`1`). However, the one with IP address `192.0.0.1` is marked as a `backup` server and does not receive requests unless both of the other servers are unavailable. With this configuration of weights, out of every `6` requests, `5` are sent to **backend1.example.com** and `1` to **backend2.example.com**. ## Server Slow-Start -The server slow‑start feature prevents a recently recovered server from being overwhelmed by connections, which may time out and cause the server to be marked as failed again. +The server slow‑start feature prevents a recently recovered server from being overwhelmed by connections, which may time out and cause the server to be marked as failed again. This feature is only available in NGINX Plus. -In NGINX Plus, slow‑start allows an upstream server to gradually recover its weight from `0` to its nominal value after it has been recovered or became available. This can be done with the [`slow_start`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameter to the `server` directive: +Slow‑start allows an upstream server to gradually recover its weight from `0` to its nominal value after it has recovered or become available. This can be done with the [`slow_start`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameter to the `server` directive: ```nginx upstream backend { @@ -189,19 +193,19 @@ upstream backend { } ``` -The time value (here, `30` seconds) sets the time during which NGINX Plus ramps up the number of connections to the server to the full value. +The time value (here, `30` seconds) sets the time during which NGINX Plus ramps up the number of connections to the server to the full value. If no value is specified, it defaults to `0`, which disables slow-start. -Note that if there is only a single server in a group, the [`max_fails`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails), [`fail_timeout`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout), and [`slow_start`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameters to the `server` directive are ignored and the server is never considered unavailable. +> **Note:** If there is only a single server in a group, the [`max_fails`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails), [`fail_timeout`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout), and [`slow_start`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameters to the `server` directive are ignored, and the server is never considered unavailable. -## Enabling Session Persistence +## Session Persistence Session persistence means that NGINX Plus identifies user sessions and routes all requests in a given session to the same upstream server. NGINX Plus supports three session persistence methods. The methods are set with the [`sticky`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky) directive. (For session persistence with NGINX Open Source, use the `hash` or `ip_hash` directive as described [above](#method).) -- [Sticky cookie](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie) – NGINX Plus adds a session cookie to the first response from the upstream group and identifies the server that sent the response. The client's next request contains the cookie value and NGINX Plus route the request to the upstream server that responded to the first request: +- [Sticky cookie](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie) – NGINX Plus adds a session cookie to the first response from the upstream group and identifies the server that sent the response. The client's next request contains the cookie value, and NGINX Plus routes the request to the upstream server that responded to the first request. This is the simplest session persistence method. ```nginx upstream backend { @@ -211,9 +215,9 @@ NGINX Plus supports three session persistence methods. The methods are set with } ``` - In the example, the `srv_id` parameter sets the name of the cookie. The optional `expires` parameter sets the time for the browser to keep the cookie (here, `1` hour). The optional `domain` parameter defines the domain for which the cookie is set, and the optional `path` parameter defines the path for which the cookie is set. This is the simplest session persistence method. + In the example above, the `srv_id` parameter sets the name of the cookie. The optional `expires` parameter sets the time for the browser to keep the cookie (here, `1` hour). The optional `domain` parameter defines the domain for which the cookie is set, and the optional `path` parameter defines the path for which the cookie is set. -- [Sticky route](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_route) – NGINX Plus assigns a “route” to the client when it receives the first request. All subsequent requests are compared to the [`route`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#route) parameter of the `server` directive to identify the server to which the request is proxied. The route information is taken from either a cookie or the request URI. +- [Sticky route](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_route) – Use the [`route`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#route) parameter to the `server` directive to assign a route identifier to each server. NGINX Plus assigns one of the route identifiers to the client when it receives the client's first request. Subsequent requests then compare these route identifiers, which are either stored in a cookie or passed in the request URI. ```nginx upstream backend { @@ -222,8 +226,9 @@ NGINX Plus supports three session persistence methods. The methods are set with sticky route $route_cookie $route_uri; } ``` - -- [Sticky learn](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_learn) method – NGINX Plus first finds session identifiers by inspecting requests and responses. Then NGINX Plus “learns” which upstream server corresponds to which session identifier. Generally, these identifiers are passed in a HTTP cookie. If a request contains a session identifier already “learned”, NGINX Plus forwards the request to the corresponding server: + In the example above, the session cookie is checked first for the route identifier. If there is none, then the URI is checked second. + +- [Sticky learn](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_learn) – NGINX Plus first finds session identifiers by inspecting requests and responses. Then NGINX Plus “learns” which upstream server corresponds to which session identifier. Generally, these identifiers are passed in a HTTP cookie. If a request contains a session identifier already “learned”, NGINX Plus forwards the request to the corresponding server: ```nginx upstream backend { @@ -266,11 +271,15 @@ NGINX Plus supports three session persistence methods. The methods are set with -## Limiting the Number of Connections +## Limit the Number of Connections + +With NGINX Plus, it is possible to limit the number of active connections to an upstream server. -With NGINX Plus, it is possible to limit the number of active connections to an upstream server by specifying the maximum number with the [`max_conns`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter. +The [`max_conns`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter sets the maximum number of connections to the upstream server. -If the `max_conns` limit has been reached, the request is placed in a queue for further processing, provided that the [`queue`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue) directive is also included to set the maximum number of requests that can be simultaneously in the queue: +The [`queue`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue) directive allows excess connections to be held in a queue. It requires a maxmum number for the queue and a timeout. + +If the `max_conns` limit has been reached, the request is placed in a queue for further processing. If the queue limit has been reached, if no queue directive is specified, or if a client reaches the `timeout` time in queue, the client will receive an error. ```nginx upstream backend { @@ -280,13 +289,11 @@ upstream backend { } ``` -If the queue is filled up with requests or the upstream server cannot be selected during the timeout specified by the optional `timeout` parameter, the client receives an error. - -Note that the `max_conns` limit is ignored if there are idle [`keepalive`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) connections opened in other [`worker processes`](https://nginx.org/en/docs/ngx_core_module.html#worker_processes). As a result, the total number of connections to the server might exceed the `max_conns` value in a configuration where the memory is [shared with multiple worker processes](#zone). +> **Note:** The `max_conns` limit is ignored if there are idle [`keepalive`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) connections opened in other [`worker processes`](https://nginx.org/en/docs/ngx_core_module.html#worker_processes). As a result, the total number of connections to the server might exceed the `max_conns` value in a configuration where the memory is [shared with multiple worker processes](#zone). -## Configuring Health Checks +## Configure Health Checks NGINX can continually test your HTTP upstream servers, avoid the servers that have failed, and gracefully add the recovered servers into the load‑balanced group. @@ -294,7 +301,7 @@ See [HTTP Health Checks]({{< ref "nginx/admin-guide/load-balancer/http-health-ch -## Sharing Data with Multiple Worker Processes +## Share Data with Multiple Worker Processes If an [`upstream`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) block does not include the [`zone`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) directive, each worker process keeps its own copy of the server group configuration and maintains its own set of related counters. The counters include the current number of connections to each server in the group and the number of failed attempts to pass a request to a server. As a result, the server group configuration cannot be modified dynamically. @@ -308,7 +315,7 @@ Similarly, the [Least Connections](#method) load‑balancing method might not wo -### Setting the Zone Size +### Set the Zone Size It is not possible to recommend an ideal memory‑zone size, because usage patterns vary widely. The required amount of memory is determined by which features (such as [session persistence](#sticky), [health checks](#health_active), or [DNS re‑resolving](#resolve)) are enabled and how the upstream servers are identified. @@ -320,7 +327,7 @@ As an example, with the `sticky_route` session persistence method and a single h -## Configuring HTTP Load Balancing Using DNS +## Configure HTTP Load Balancing Using DNS The configuration of a server group can be modified at runtime using DNS.