Skip to content

Commit c1ca166

Browse files
LorcanMcVeighDean-Coakleypleshakov
authored
Add configurable timeouts to TransportServer
Co-authored-by: Dean Coakley <[email protected]> Co-authored-by: Michael Pleshakov <[email protected]>
1 parent 48bde85 commit c1ca166

File tree

14 files changed

+318
-52
lines changed

14 files changed

+318
-52
lines changed

‎deployments/common/crds-v1beta1/k8s.nginx.org_transportservers.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,24 @@ spec:
4949
type: string
5050
protocol:
5151
type: string
52+
sessionParameters:
53+
description: SessionParameters defines session parameters.
54+
type: object
55+
properties:
56+
timeout:
57+
type: string
5258
upstreamParameters:
5359
description: UpstreamParameters defines parameters for an upstream.
5460
type: object
5561
properties:
62+
connectTimeout:
63+
type: string
64+
nextUpstream:
65+
type: boolean
66+
nextUpstreamTimeout:
67+
type: string
68+
nextUpstreamTries:
69+
type: integer
5670
udpRequests:
5771
type: integer
5872
udpResponses:

‎deployments/common/crds/k8s.nginx.org_transportservers.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,10 +50,24 @@ spec:
5050
type: string
5151
protocol:
5252
type: string
53+
sessionParameters:
54+
description: SessionParameters defines session parameters.
55+
type: object
56+
properties:
57+
timeout:
58+
type: string
5359
upstreamParameters:
5460
description: UpstreamParameters defines parameters for an upstream.
5561
type: object
5662
properties:
63+
connectTimeout:
64+
type: string
65+
nextUpstream:
66+
type: boolean
67+
nextUpstreamTimeout:
68+
type: string
69+
nextUpstreamTries:
70+
type: integer
5771
udpRequests:
5872
type: integer
5973
udpResponses:

‎deployments/helm-chart/crds/k8s.nginx.org_transportservers.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,24 @@ spec:
4949
type: string
5050
protocol:
5151
type: string
52+
sessionParameters:
53+
description: SessionParameters defines session parameters.
54+
type: object
55+
properties:
56+
timeout:
57+
type: string
5258
upstreamParameters:
5359
description: UpstreamParameters defines parameters for an upstream.
5460
type: object
5561
properties:
62+
connectTimeout:
63+
type: string
64+
nextUpstream:
65+
type: boolean
66+
nextUpstreamTimeout:
67+
type: string
68+
nextUpstreamTries:
69+
type: integer
5670
udpRequests:
5771
type: integer
5872
udpResponses:

‎docs-web/configuration/transportserver-resource.md‎

Lines changed: 45 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ This document is the reference documentation for the TransportServer resource. T
1515
- [Listener](#listener)
1616
- [Upstream](#upstream)
1717
- [UpstreamParameters](#upstreamparameters)
18+
- [SessionParameters](#sessionparameters)
1819
- [Action](#action)
1920
- [Using TransportServer](#using-transportserver)
2021
- [Validation](#validation)
@@ -182,11 +183,15 @@ port: 8443
182183
183184
### UpstreamParameters
184185
185-
The upstream parameters define various parameters for the upstreams. For now, only UDP-related parameters are supported:
186+
The upstream parameters define various parameters for the upstreams:
186187
```yaml
187188
upstreamParameters:
188189
udpRequests: 1
189190
udpResponses: 1
191+
connectTimeout: 60s
192+
nextUpstream: true
193+
nextUpstreamTimeout: 50s
194+
nextUpstreamTries: 1
190195
```
191196
192197
```eval_rst
@@ -205,6 +210,44 @@ upstreamParameters:
205210
- The number of datagrams expected from the proxied server in response to a client datagram. See the `proxy_responses <https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_responses>`_ directive. By default, the number of datagrams is not limited.
206211
- ``int``
207212
- No
213+
* - ``connectTimeout``
214+
- The timeout for establishing a connection with a proxied server. See the `proxy_connect_timeout <http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_connect_timeout>`_ directive. The default is ``60s``.
215+
- ``string``
216+
- No
217+
* - ``nextUpstream``
218+
- If a connection to the proxied server cannot be established, determines whether a client connection will be passed to the next server. See the `proxy_next_upstream <http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream>`_ directive. The default is ``true``.
219+
- bool
220+
- No
221+
* - ``nextUpstreamTries``
222+
- The number of tries for passing a connection to the next server. See the `proxy_next_upstream_tries <http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_tries>`_ directive. The default is ``0``.
223+
- ``int``
224+
- No
225+
* - ``nextUpstreamTimeout``
226+
- The time allowed to pass a connection to the next server. See the `proxy_next_upstream_timeout <http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_timeout>`_ directive. The default us ``0``.
227+
- ``string``
228+
- No
229+
```
230+
231+
### SessionParameters
232+
233+
The session parameters define various parameters for TCP connections and UDP sessions.
234+
```yaml
235+
sessionParameters:
236+
timeout: 50s
237+
```
238+
239+
```eval_rst
240+
.. list-table::
241+
:header-rows: 1
242+
243+
* - Field
244+
- Description
245+
- Type
246+
- Required
247+
* - ``timeout``
248+
- The timeout between two succesive read or write operations on client or proxied server connections. See `proxy_timeout <http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout>`_ directive. The default is ``10m``.
249+
- ``string``
250+
- No
208251
```
209252
210253
### Action
@@ -312,4 +355,4 @@ The [ConfigMap](/nginx-ingress-controller/configuration/global-configuration/con
312355
As of Release 1.7, the TransportServer resource is a preview feature. Currently, it comes with the following limitations:
313356
* When using TLS Passthrough, it is not possible to configure [Proxy Protocol](https://github.com/nginxinc/kubernetes-ingress/tree/master/examples/proxy-protocol) for port 443 both for regular HTTPS and TLS Passthrough traffic.
314357
* If multiple TCP (or UDP) TransportServers reference the same listener, only one of them will receive the traffic. Moreover, until there is only one TransportServer, NGINX will fail to reload. If this happens, the IC will report a warning event with the `AddedOrUpdatedWithError` reason for the resource, which caused the problem, and also report the error in the logs.
315-
* If multiple TLS Passthrough TransportServers have the same hostname, only one of them will receive the traffic. If this happens, the IC will report a warning in the logs like `host "app.example.com" is used by more than one TransportServers`.
358+
* If multiple TLS Passthrough TransportServers have the same hostname, only one of them will receive the traffic. If this happens, the IC will report a warning in the logs like `host "app.example.com" is used by more than one TransportServers`.

‎internal/configs/transportserver.go‎

Lines changed: 32 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,27 @@ func generateTransportServerConfig(transportServerEx *TransportServerEx, listene
3535
upstreams := generateStreamUpstreams(transportServerEx, upstreamNamer, isPlus)
3636

3737
var proxyRequests, proxyResponses *int
38+
var connectTimeout, nextUpstreamTimeout string
39+
var nextUpstream bool
40+
var nextUpstreamTries int
3841
if transportServerEx.TransportServer.Spec.UpstreamParameters != nil {
3942
proxyRequests = transportServerEx.TransportServer.Spec.UpstreamParameters.UDPRequests
4043
proxyResponses = transportServerEx.TransportServer.Spec.UpstreamParameters.UDPResponses
44+
45+
nextUpstream = transportServerEx.TransportServer.Spec.UpstreamParameters.NextUpstream
46+
if nextUpstream {
47+
nextUpstreamTries = transportServerEx.TransportServer.Spec.UpstreamParameters.NextUpstreamTries
48+
nextUpstreamTimeout = transportServerEx.TransportServer.Spec.UpstreamParameters.NextUpstreamTimeout
49+
}
50+
51+
connectTimeout = transportServerEx.TransportServer.Spec.UpstreamParameters.ConnectTimeout
4152
}
53+
54+
var proxyTimeout string
55+
if transportServerEx.TransportServer.Spec.SessionParameters != nil {
56+
proxyTimeout = transportServerEx.TransportServer.Spec.SessionParameters.Timeout
57+
}
58+
4259
statusZone := ""
4360
if transportServerEx.TransportServer.Spec.Listener.Name == conf_v1alpha1.TLSPassthroughListenerName {
4461
statusZone = transportServerEx.TransportServer.Spec.Host
@@ -48,16 +65,21 @@ func generateTransportServerConfig(transportServerEx *TransportServerEx, listene
4865

4966
return version2.TransportServerConfig{
5067
Server: version2.StreamServer{
51-
TLSPassthrough: transportServerEx.TransportServer.Spec.Listener.Name == conf_v1alpha1.TLSPassthroughListenerName,
52-
UnixSocket: generateUnixSocket(transportServerEx),
53-
Port: listenerPort,
54-
UDP: transportServerEx.TransportServer.Spec.Listener.Protocol == "UDP",
55-
StatusZone: statusZone,
56-
ProxyRequests: proxyRequests,
57-
ProxyResponses: proxyResponses,
58-
ProxyPass: upstreamNamer.GetNameForUpstream(transportServerEx.TransportServer.Spec.Action.Pass),
59-
Name: transportServerEx.TransportServer.Name,
60-
Namespace: transportServerEx.TransportServer.Namespace,
68+
TLSPassthrough: transportServerEx.TransportServer.Spec.Listener.Name == conf_v1alpha1.TLSPassthroughListenerName,
69+
UnixSocket: generateUnixSocket(transportServerEx),
70+
Port: listenerPort,
71+
UDP: transportServerEx.TransportServer.Spec.Listener.Protocol == "UDP",
72+
StatusZone: statusZone,
73+
ProxyRequests: proxyRequests,
74+
ProxyResponses: proxyResponses,
75+
ProxyPass: upstreamNamer.GetNameForUpstream(transportServerEx.TransportServer.Spec.Action.Pass),
76+
Name: transportServerEx.TransportServer.Name,
77+
Namespace: transportServerEx.TransportServer.Namespace,
78+
ProxyConnectTimeout: generateString(connectTimeout, "60s"),
79+
ProxyTimeout: generateString(proxyTimeout, "10m"),
80+
ProxyNextUpstream: nextUpstream,
81+
ProxyNextUpstreamTimeout: generateString(nextUpstreamTimeout, "0"),
82+
ProxyNextUpstreamTries: nextUpstreamTries,
6183
},
6284
Upstreams: upstreams,
6385
}

‎internal/configs/transportserver_test.go‎

Lines changed: 56 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,13 @@ func TestGenerateTransportServerConfigForTCP(t *testing.T) {
8080
Port: 5001,
8181
},
8282
},
83+
UpstreamParameters: &conf_v1alpha1.UpstreamParameters{
84+
ConnectTimeout: "30s",
85+
NextUpstream: false,
86+
},
87+
SessionParameters: &conf_v1alpha1.SessionParameters{
88+
Timeout: "50s",
89+
},
8390
Action: &conf_v1alpha1.Action{
8491
Pass: "tcp-app",
8592
},
@@ -112,12 +119,17 @@ func TestGenerateTransportServerConfigForTCP(t *testing.T) {
112119
},
113120
},
114121
Server: version2.StreamServer{
115-
Port: 2020,
116-
UDP: false,
117-
StatusZone: "tcp-listener",
118-
ProxyPass: "ts_default_tcp-server_tcp-app",
119-
Name: "tcp-server",
120-
Namespace: "default",
122+
Port: 2020,
123+
UDP: false,
124+
StatusZone: "tcp-listener",
125+
ProxyPass: "ts_default_tcp-server_tcp-app",
126+
Name: "tcp-server",
127+
Namespace: "default",
128+
ProxyConnectTimeout: "30s",
129+
ProxyNextUpstream: false,
130+
ProxyNextUpstreamTries: 0,
131+
ProxyNextUpstreamTimeout: "0",
132+
ProxyTimeout: "50s",
121133
},
122134
}
123135

@@ -148,6 +160,12 @@ func TestGenerateTransportServerConfigForTLSPasstrhough(t *testing.T) {
148160
Port: 5001,
149161
},
150162
},
163+
UpstreamParameters: &conf_v1alpha1.UpstreamParameters{
164+
ConnectTimeout: "30s",
165+
NextUpstream: false,
166+
NextUpstreamTries: 0,
167+
NextUpstreamTimeout: "",
168+
},
151169
Action: &conf_v1alpha1.Action{
152170
Pass: "tcp-app",
153171
},
@@ -180,14 +198,19 @@ func TestGenerateTransportServerConfigForTLSPasstrhough(t *testing.T) {
180198
},
181199
},
182200
Server: version2.StreamServer{
183-
TLSPassthrough: true,
184-
UnixSocket: "unix:/var/lib/nginx/passthrough-default_tcp-server.sock",
185-
Port: 2020,
186-
UDP: false,
187-
StatusZone: "example.com",
188-
ProxyPass: "ts_default_tcp-server_tcp-app",
189-
Name: "tcp-server",
190-
Namespace: "default",
201+
TLSPassthrough: true,
202+
UnixSocket: "unix:/var/lib/nginx/passthrough-default_tcp-server.sock",
203+
Port: 2020,
204+
UDP: false,
205+
StatusZone: "example.com",
206+
ProxyPass: "ts_default_tcp-server_tcp-app",
207+
Name: "tcp-server",
208+
Namespace: "default",
209+
ProxyConnectTimeout: "30s",
210+
ProxyNextUpstream: false,
211+
ProxyNextUpstreamTimeout: "0",
212+
ProxyNextUpstreamTries: 0,
213+
ProxyTimeout: "10m",
191214
},
192215
}
193216

@@ -221,8 +244,12 @@ func TestGenerateTransportServerConfigForUDP(t *testing.T) {
221244
},
222245
},
223246
UpstreamParameters: &conf_v1alpha1.UpstreamParameters{
224-
UDPRequests: &udpRequests,
225-
UDPResponses: &udpResponses,
247+
UDPRequests: &udpRequests,
248+
UDPResponses: &udpResponses,
249+
ConnectTimeout: "30s",
250+
NextUpstream: true,
251+
NextUpstreamTimeout: "",
252+
NextUpstreamTries: 0,
226253
},
227254
Action: &conf_v1alpha1.Action{
228255
Pass: "udp-app",
@@ -256,14 +283,19 @@ func TestGenerateTransportServerConfigForUDP(t *testing.T) {
256283
},
257284
},
258285
Server: version2.StreamServer{
259-
Port: 2020,
260-
UDP: true,
261-
StatusZone: "udp-listener",
262-
ProxyRequests: &udpRequests,
263-
ProxyResponses: &udpResponses,
264-
ProxyPass: "ts_default_udp-server_udp-app",
265-
Name: "udp-server",
266-
Namespace: "default",
286+
Port: 2020,
287+
UDP: true,
288+
StatusZone: "udp-listener",
289+
ProxyRequests: &udpRequests,
290+
ProxyResponses: &udpResponses,
291+
ProxyPass: "ts_default_udp-server_udp-app",
292+
Name: "udp-server",
293+
Namespace: "default",
294+
ProxyConnectTimeout: "30s",
295+
ProxyNextUpstream: true,
296+
ProxyNextUpstreamTimeout: "0",
297+
ProxyNextUpstreamTries: 0,
298+
ProxyTimeout: "10m",
267299
},
268300
}
269301

‎internal/configs/version2/nginx-plus.transportserver.tmpl‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,4 +29,13 @@ server {
2929
{{ end }}
3030

3131
proxy_pass {{ $s.ProxyPass }};
32+
33+
proxy_timeout {{ $s.ProxyTimeout }};
34+
proxy_connect_timeout {{ $s.ProxyConnectTimeout }};
35+
36+
{{ if $s.ProxyNextUpstream }}
37+
proxy_next_upstream on;
38+
proxy_next_upstream_timeout {{ $s.ProxyNextUpstreamTimeout }};
39+
proxy_next_upstream_tries {{ $s.ProxyNextUpstreamTries }};
40+
{{ end }}
3241
}

‎internal/configs/version2/nginx.transportserver.tmpl‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,4 +27,13 @@ server {
2727
{{ end }}
2828

2929
proxy_pass {{ $s.ProxyPass }};
30+
31+
proxy_timeout {{ $s.ProxyTimeout }};
32+
proxy_connect_timeout {{ $s.ProxyConnectTimeout }};
33+
34+
{{ if $s.ProxyNextUpstream }}
35+
proxy_next_upstream on;
36+
proxy_next_upstream_timeout {{ $s.ProxyNextUpstreamTimeout }};
37+
proxy_next_upstream_tries {{ $s.ProxyNextUpstreamTries }};
38+
{{ end }}
3039
}

‎internal/configs/version2/stream.go‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -20,16 +20,21 @@ type StreamUpstreamServer struct {
2020

2121
// StreamServer defines a server in the stream module.
2222
type StreamServer struct {
23-
TLSPassthrough bool
24-
UnixSocket string
25-
Port int
26-
UDP bool
27-
StatusZone string
28-
ProxyRequests *int
29-
ProxyResponses *int
30-
ProxyPass string
31-
Name string
32-
Namespace string
23+
TLSPassthrough bool
24+
UnixSocket string
25+
Port int
26+
UDP bool
27+
StatusZone string
28+
ProxyRequests *int
29+
ProxyResponses *int
30+
ProxyPass string
31+
Name string
32+
Namespace string
33+
ProxyTimeout string
34+
ProxyConnectTimeout string
35+
ProxyNextUpstream bool
36+
ProxyNextUpstreamTimeout string
37+
ProxyNextUpstreamTries int
3338
}
3439

3540
// TLSPassthroughHostsConfig defines a mapping between TLS Passthrough hosts and the corresponding unix sockets.

0 commit comments

Comments
 (0)