Skip to content
This repository was archived by the owner on Mar 26, 2024. It is now read-only.

Commit 13e9b31

Browse files
FizzadarFizzadar
committed
Merge tag 'v1.89.0' into merge-1.89
No significant changes since 1.89.0rc1. - Add Unix Socket support for HTTP Replication Listeners. [Document and provide usage instructions](https://matrix-org.github.io/synapse/v1.89/usage/configuration/config_documentation.html#listeners) for utilizing Unix sockets in Synapse. Contributed by Jason Little. ([\matrix-org#15708](matrix-org#15708), [\matrix-org#15924](matrix-org#15924)) - Allow `+` in Matrix IDs, per [MSC4009](matrix-org/matrix-spec-proposals#4009). ([\matrix-org#15911](matrix-org#15911)) - Support room version 11 from [MSC3820](matrix-org/matrix-spec-proposals#3820). ([\matrix-org#15912](matrix-org#15912)) - Allow configuring the set of workers to proxy outbound federation traffic through via `outbound_federation_restricted_to`. ([\matrix-org#15913](matrix-org#15913), [\matrix-org#15969](matrix-org#15969)) - Implement [MSC3814](matrix-org/matrix-spec-proposals#3814), dehydrated devices v2/shrivelled sessions and move [MSC2697](matrix-org/matrix-spec-proposals#2697) behind a config flag. Contributed by Nico from Famedly, H-Shay and poljar. ([\matrix-org#15929](matrix-org#15929)) - Fix a long-standing bug where remote invites weren't correctly pushed. ([\matrix-org#15820](matrix-org#15820)) - Fix background schema updates failing over a large upgrade gap. ([\matrix-org#15887](matrix-org#15887)) - Fix a bug introduced in 1.86.0 where Synapse starting with an empty `experimental_features` configuration setting. ([\matrix-org#15925](matrix-org#15925)) - Fixed deploy annotations in the provided Grafana dashboard config, so that it shows for any homeserver and not just matrix.org. Contributed by @wrjlewis. ([\matrix-org#15957](matrix-org#15957)) - Ensure a long state res does not starve CPU by occasionally yielding to the reactor. ([\matrix-org#15960](matrix-org#15960)) - Properly handle redactions of creation events. ([\matrix-org#15973](matrix-org#15973)) - Fix a bug where resyncing stale device lists could block responding to federation transactions, and thus delay receiving new data from the remote server. ([\matrix-org#15975](matrix-org#15975)) - Better clarify how to run a worker instance (pass both configs). ([\matrix-org#15921](matrix-org#15921)) - Improve [the documentation](https://matrix-org.github.io/synapse/v1.89/admin_api/user_admin_api.html#login-as-a-user) for the login as a user admin API. ([\matrix-org#15938](matrix-org#15938)) - Fix broken Arch Linux package link. Contributed by @SnipeXandrej. ([\matrix-org#15981](matrix-org#15981)) - Remove support for calling the `/register` endpoint with an unspecced `user` property for application services. ([\matrix-org#15928](matrix-org#15928)) - Mark `get_user_in_directory` private since it is only used in tests. Also remove the cache from it. ([\matrix-org#15884](matrix-org#15884)) - Document which Python version runs on a given Linux distribution so we can more easily clean up later. ([\matrix-org#15909](matrix-org#15909)) - Add details to warning in log when we fail to fetch an alias. ([\matrix-org#15922](matrix-org#15922)) - Remove unneeded `__init__`. ([\matrix-org#15926](matrix-org#15926)) - Fix bug with read/write lock implementation. This is currently unused so has no observable effects. ([\matrix-org#15933](matrix-org#15933), [\matrix-org#15958](matrix-org#15958)) - Unbreak the nix development environment by pinning the Rust version to 1.70.0. ([\matrix-org#15940](matrix-org#15940)) - Update presence metrics to differentiate remote vs local users. ([\matrix-org#15952](matrix-org#15952)) - Stop reading from column `user_id` of table `profiles`. ([\matrix-org#15955](matrix-org#15955)) - Build packages for Debian Trixie. ([\matrix-org#15961](matrix-org#15961)) - Reduce the amount of state we pull out. ([\matrix-org#15968](matrix-org#15968)) - Speed up updating state in large rooms. ([\matrix-org#15971](matrix-org#15971)) * Bump anyhow from 1.0.71 to 1.0.72. ([\matrix-org#15949](matrix-org#15949)) * Bump click from 8.1.3 to 8.1.6. ([\matrix-org#15984](matrix-org#15984)) * Bump cryptography from 41.0.1 to 41.0.2. ([\matrix-org#15943](matrix-org#15943)) * Bump jsonschema from 4.17.3 to 4.18.3. ([\matrix-org#15948](matrix-org#15948)) * Bump pillow from 9.4.0 to 10.0.0. ([\matrix-org#15986](matrix-org#15986)) * Bump prometheus-client from 0.17.0 to 0.17.1. ([\matrix-org#15945](matrix-org#15945)) * Bump pydantic from 1.10.10 to 1.10.11. ([\matrix-org#15946](matrix-org#15946)) * Bump pygithub from 1.58.2 to 1.59.0. ([\matrix-org#15834](matrix-org#15834)) * Bump pyo3-log from 0.8.2 to 0.8.3. ([\matrix-org#15951](matrix-org#15951)) * Bump sentry-sdk from 1.26.0 to 1.28.1. ([\matrix-org#15985](matrix-org#15985)) * Bump serde_json from 1.0.100 to 1.0.103. ([\matrix-org#15950](matrix-org#15950)) * Bump types-pillow from 9.5.0.4 to 10.0.0.1. ([\matrix-org#15932](matrix-org#15932)) * Bump types-requests from 2.31.0.1 to 2.31.0.2. ([\matrix-org#15983](matrix-org#15983)) * Bump typing-extensions from 4.5.0 to 4.7.1. ([\matrix-org#15947](matrix-org#15947)) # -----BEGIN PGP SIGNATURE----- # # iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmTI2e4ACgkQkD7OEIo5 # 3t2x1RAAohu1Rmjv0mOqFR4P1YZpA5RFbYajcyq77n/ciDKSM1dqBelONqKOq2A9 # uGbVNm6rC+EFwIl5MF5TrFdsDQHvGcRgW6NpQDZ+uIUOYizjZH1g37BoNPLlGYQx # fmKG7/XqdWhSc5tHN9HsRHyHKmsndebjXoUCPKmieGZa1GLXvGwrNkWQlEpwd9Qu # mj3uewJxLFGgIIAOiplJ4UO8FaCbMD+By27hSiWtVsLT6pyav4HC2P8RQD1iv0jW # OXNHvEWyqfBPlsPOkCD4nQZrmZqa5GWLYfBm8zFgIBxNy+e33C07L4bO+QdCE86v # /SUKug/0nsp66jSZst1fM/M2ssXvjU+LNO9fqonOCZ4TiJ4i/yoa8AvmcAg5hy7C # HR9IBp9cMrQ2u1y2/knxF657AGHxgXEltgw0PDvZHowqsqoSb+5HWl0zv1wnVjMa # 2QYLKWPBk/AdlHkmC3S4/+gfVZVsT2RSBP3JUCbFyOqug9vXFvSGTfH07Lk4PDI3 # o5idBzumvyonsuC2ypkzlj49FAj21l/8DInxEpY9JcHdVncLWvu9gmLd+H7GY7H7 # ODa2gOynrsSGVH7IpOl6dpw/GH6R8ZlfHl87bFslOqVObBxquL/ODIoFOgld+MpT # YYXp+0tW564mg+AYw3+eo44JTq0lKh7eyENP3SqKN/Z8ssQL97c= # =Ar/g # -----END PGP SIGNATURE----- # gpg: Signature made Tue Aug 1 11:09:50 2023 BST # gpg: using RSA key D79D3CA0B61429A8A760525A903ECE108A39DEDD # gpg: key 903ECE108A39DEDD: new key but contains no user ID - skipped # gpg: Total number processed: 1 # gpg: w/o user IDs: 1 # gpg: Can't check signature: No public key # Conflicts: # poetry.lock # synapse/http/site.py # synapse/storage/databases/main/roommember.py
2 parents ab86c68 + 190c990 commit 13e9b31

File tree

103 files changed

+2884
-878
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

103 files changed

+2884
-878
lines changed

‎CHANGES.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,69 @@
1+
# Synapse 1.89.0 (2023-08-01)
2+
3+
No significant changes since 1.89.0rc1.
4+
5+
6+
# Synapse 1.89.0rc1 (2023-07-25)
7+
8+
### Features
9+
10+
- Add Unix Socket support for HTTP Replication Listeners. [Document and provide usage instructions](https://matrix-org.github.io/synapse/v1.89/usage/configuration/config_documentation.html#listeners) for utilizing Unix sockets in Synapse. Contributed by Jason Little. ([\#15708](https://github.com/matrix-org/synapse/issues/15708), [\#15924](https://github.com/matrix-org/synapse/issues/15924))
11+
- Allow `+` in Matrix IDs, per [MSC4009](https://github.com/matrix-org/matrix-spec-proposals/pull/4009). ([\#15911](https://github.com/matrix-org/synapse/issues/15911))
12+
- Support room version 11 from [MSC3820](https://github.com/matrix-org/matrix-spec-proposals/pull/3820). ([\#15912](https://github.com/matrix-org/synapse/issues/15912))
13+
- Allow configuring the set of workers to proxy outbound federation traffic through via `outbound_federation_restricted_to`. ([\#15913](https://github.com/matrix-org/synapse/issues/15913), [\#15969](https://github.com/matrix-org/synapse/issues/15969))
14+
- Implement [MSC3814](https://github.com/matrix-org/matrix-spec-proposals/pull/3814), dehydrated devices v2/shrivelled sessions and move [MSC2697](https://github.com/matrix-org/matrix-spec-proposals/pull/2697) behind a config flag. Contributed by Nico from Famedly, H-Shay and poljar. ([\#15929](https://github.com/matrix-org/synapse/issues/15929))
15+
16+
### Bugfixes
17+
18+
- Fix a long-standing bug where remote invites weren't correctly pushed. ([\#15820](https://github.com/matrix-org/synapse/issues/15820))
19+
- Fix background schema updates failing over a large upgrade gap. ([\#15887](https://github.com/matrix-org/synapse/issues/15887))
20+
- Fix a bug introduced in 1.86.0 where Synapse starting with an empty `experimental_features` configuration setting. ([\#15925](https://github.com/matrix-org/synapse/issues/15925))
21+
- Fixed deploy annotations in the provided Grafana dashboard config, so that it shows for any homeserver and not just matrix.org. Contributed by @wrjlewis. ([\#15957](https://github.com/matrix-org/synapse/issues/15957))
22+
- Ensure a long state res does not starve CPU by occasionally yielding to the reactor. ([\#15960](https://github.com/matrix-org/synapse/issues/15960))
23+
- Properly handle redactions of creation events. ([\#15973](https://github.com/matrix-org/synapse/issues/15973))
24+
- Fix a bug where resyncing stale device lists could block responding to federation transactions, and thus delay receiving new data from the remote server. ([\#15975](https://github.com/matrix-org/synapse/issues/15975))
25+
26+
### Improved Documentation
27+
28+
- Better clarify how to run a worker instance (pass both configs). ([\#15921](https://github.com/matrix-org/synapse/issues/15921))
29+
- Improve [the documentation](https://matrix-org.github.io/synapse/v1.89/admin_api/user_admin_api.html#login-as-a-user) for the login as a user admin API. ([\#15938](https://github.com/matrix-org/synapse/issues/15938))
30+
- Fix broken Arch Linux package link. Contributed by @SnipeXandrej. ([\#15981](https://github.com/matrix-org/synapse/issues/15981))
31+
32+
### Deprecations and Removals
33+
34+
- Remove support for calling the `/register` endpoint with an unspecced `user` property for application services. ([\#15928](https://github.com/matrix-org/synapse/issues/15928))
35+
36+
### Internal Changes
37+
38+
- Mark `get_user_in_directory` private since it is only used in tests. Also remove the cache from it. ([\#15884](https://github.com/matrix-org/synapse/issues/15884))
39+
- Document which Python version runs on a given Linux distribution so we can more easily clean up later. ([\#15909](https://github.com/matrix-org/synapse/issues/15909))
40+
- Add details to warning in log when we fail to fetch an alias. ([\#15922](https://github.com/matrix-org/synapse/issues/15922))
41+
- Remove unneeded `__init__`. ([\#15926](https://github.com/matrix-org/synapse/issues/15926))
42+
- Fix bug with read/write lock implementation. This is currently unused so has no observable effects. ([\#15933](https://github.com/matrix-org/synapse/issues/15933), [\#15958](https://github.com/matrix-org/synapse/issues/15958))
43+
- Unbreak the nix development environment by pinning the Rust version to 1.70.0. ([\#15940](https://github.com/matrix-org/synapse/issues/15940))
44+
- Update presence metrics to differentiate remote vs local users. ([\#15952](https://github.com/matrix-org/synapse/issues/15952))
45+
- Stop reading from column `user_id` of table `profiles`. ([\#15955](https://github.com/matrix-org/synapse/issues/15955))
46+
- Build packages for Debian Trixie. ([\#15961](https://github.com/matrix-org/synapse/issues/15961))
47+
- Reduce the amount of state we pull out. ([\#15968](https://github.com/matrix-org/synapse/issues/15968))
48+
- Speed up updating state in large rooms. ([\#15971](https://github.com/matrix-org/synapse/issues/15971))
49+
50+
### Updates to locked dependencies
51+
52+
* Bump anyhow from 1.0.71 to 1.0.72. ([\#15949](https://github.com/matrix-org/synapse/issues/15949))
53+
* Bump click from 8.1.3 to 8.1.6. ([\#15984](https://github.com/matrix-org/synapse/issues/15984))
54+
* Bump cryptography from 41.0.1 to 41.0.2. ([\#15943](https://github.com/matrix-org/synapse/issues/15943))
55+
* Bump jsonschema from 4.17.3 to 4.18.3. ([\#15948](https://github.com/matrix-org/synapse/issues/15948))
56+
* Bump pillow from 9.4.0 to 10.0.0. ([\#15986](https://github.com/matrix-org/synapse/issues/15986))
57+
* Bump prometheus-client from 0.17.0 to 0.17.1. ([\#15945](https://github.com/matrix-org/synapse/issues/15945))
58+
* Bump pydantic from 1.10.10 to 1.10.11. ([\#15946](https://github.com/matrix-org/synapse/issues/15946))
59+
* Bump pygithub from 1.58.2 to 1.59.0. ([\#15834](https://github.com/matrix-org/synapse/issues/15834))
60+
* Bump pyo3-log from 0.8.2 to 0.8.3. ([\#15951](https://github.com/matrix-org/synapse/issues/15951))
61+
* Bump sentry-sdk from 1.26.0 to 1.28.1. ([\#15985](https://github.com/matrix-org/synapse/issues/15985))
62+
* Bump serde_json from 1.0.100 to 1.0.103. ([\#15950](https://github.com/matrix-org/synapse/issues/15950))
63+
* Bump types-pillow from 9.5.0.4 to 10.0.0.1. ([\#15932](https://github.com/matrix-org/synapse/issues/15932))
64+
* Bump types-requests from 2.31.0.1 to 2.31.0.2. ([\#15983](https://github.com/matrix-org/synapse/issues/15983))
65+
* Bump typing-extensions from 4.5.0 to 4.7.1. ([\#15947](https://github.com/matrix-org/synapse/issues/15947))
66+
167
# Synapse 1.88.0 (2023-07-18)
268

369
This release

‎Cargo.lock‎

Lines changed: 6 additions & 6 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎contrib/grafana/synapse.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@
6363
"uid": "${DS_PROMETHEUS}"
6464
},
6565
"enable": true,
66-
"expr": "changes(process_start_time_seconds{instance=\"matrix.org\",job=~\"synapse\"}[$bucket_size]) * on (instance, job) group_left(version) synapse_build_info{instance=\"matrix.org\",job=\"synapse\"}",
66+
"expr": "changes(process_start_time_seconds{instance=\"$instance\",job=~\"synapse\"}[$bucket_size]) * on (instance, job) group_left(version) synapse_build_info{instance=\"$instance\",job=\"synapse\"}",
6767
"iconColor": "purple",
6868
"name": "deploys",
6969
"titleFormat": "Deployed {{version}}"

‎debian/changelog‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,15 @@
1+
matrix-synapse-py3 (1.89.0) stable; urgency=medium
2+
3+
* New Synapse release 1.89.0.
4+
5+
-- Synapse Packaging team <[email protected]> Tue, 01 Aug 2023 11:07:15 +0100
6+
7+
matrix-synapse-py3 (1.89.0~rc1) stable; urgency=medium
8+
9+
* New Synapse release 1.89.0rc1.
10+
11+
-- Synapse Packaging team <[email protected]> Tue, 25 Jul 2023 14:31:07 +0200
12+
113
matrix-synapse-py3 (1.88.0) stable; urgency=medium
214

315
* New Synapse release 1.88.0.

‎docker/conf-workers/nginx.conf.j2‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,11 @@ server {
3535

3636
# Send all other traffic to the main process
3737
location ~* ^(\\/_matrix|\\/_synapse) {
38+
{% if using_unix_sockets %}
39+
proxy_pass http://unix:/run/main_public.sock;
40+
{% else %}
3841
proxy_pass http://localhost:8080;
42+
{% endif %}
3943
proxy_set_header X-Forwarded-For $remote_addr;
4044
proxy_set_header X-Forwarded-Proto $scheme;
4145
proxy_set_header Host $host;

‎docker/conf-workers/shared.yaml.j2‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,9 @@
66
{% if enable_redis %}
77
redis:
88
enabled: true
9+
{% if using_unix_sockets %}
10+
path: /tmp/redis.sock
11+
{% endif %}
912
{% endif %}
1013

1114
{% if appservice_registrations is not none %}

‎docker/conf-workers/supervisord.conf.j2‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,11 @@ username=www-data
1919
autorestart=true
2020

2121
[program:redis]
22+
{% if using_unix_sockets %}
23+
command=/usr/local/bin/prefix-log /usr/local/bin/redis-server --unixsocket /tmp/redis.sock
24+
{% else %}
2225
command=/usr/local/bin/prefix-log /usr/local/bin/redis-server
26+
{% endif %}
2327
priority=1
2428
stdout_logfile=/dev/stdout
2529
stdout_logfile_maxbytes=0

‎docker/conf-workers/worker.yaml.j2‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,11 @@ worker_name: "{{ name }}"
88

99
worker_listeners:
1010
- type: http
11+
{% if using_unix_sockets %}
12+
path: "/run/worker.{{ port }}"
13+
{% else %}
1114
port: {{ port }}
15+
{% endif %}
1216
{% if listener_resources %}
1317
resources:
1418
- names:

‎docker/conf/homeserver.yaml‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,12 +36,17 @@ listeners:
3636

3737
# Allow configuring in case we want to reverse proxy 8008
3838
# using another process in the same container
39+
{% if SYNAPSE_USE_UNIX_SOCKET %}
40+
# Unix sockets don't care about TLS or IP addresses or ports
41+
- path: '/run/main_public.sock'
42+
type: http
43+
{% else %}
3944
- port: {{ SYNAPSE_HTTP_PORT or 8008 }}
4045
tls: false
4146
bind_addresses: ['::']
4247
type: http
4348
x_forwarded: false
44-
49+
{% endif %}
4550
resources:
4651
- names: [client]
4752
compress: true
@@ -57,8 +62,11 @@ database:
5762
user: "{{ POSTGRES_USER or "synapse" }}"
5863
password: "{{ POSTGRES_PASSWORD }}"
5964
database: "{{ POSTGRES_DB or "synapse" }}"
65+
{% if not SYNAPSE_USE_UNIX_SOCKET %}
66+
{# Synapse will use a default unix socket for Postgres when host/port is not specified (behavior from `psycopg2`). #}
6067
host: "{{ POSTGRES_HOST or "db" }}"
6168
port: "{{ POSTGRES_PORT or "5432" }}"
69+
{% endif %}
6270
cp_min: 5
6371
cp_max: 10
6472
{% else %}

‎docker/configure_workers_and_start.py‎

Lines changed: 78 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,9 @@
7474
MAIN_PROCESS_INSTANCE_NAME = "main"
7575
MAIN_PROCESS_LOCALHOST_ADDRESS = "127.0.0.1"
7676
MAIN_PROCESS_REPLICATION_PORT = 9093
77+
# Obviously, these would only be used with the UNIX socket option
78+
MAIN_PROCESS_UNIX_SOCKET_PUBLIC_PATH = "/run/main_public.sock"
79+
MAIN_PROCESS_UNIX_SOCKET_PRIVATE_PATH = "/run/main_private.sock"
7780

7881
# A simple name used as a placeholder in the WORKERS_CONFIG below. This will be replaced
7982
# during processing with the name of the worker.
@@ -407,11 +410,15 @@ def add_worker_roles_to_shared_config(
407410
)
408411

409412
# Map of stream writer instance names to host/ports combos
410-
instance_map[worker_name] = {
411-
"host": "localhost",
412-
"port": worker_port,
413-
}
414-
413+
if os.environ.get("SYNAPSE_USE_UNIX_SOCKET", False):
414+
instance_map[worker_name] = {
415+
"path": f"/run/worker.{worker_port}",
416+
}
417+
else:
418+
instance_map[worker_name] = {
419+
"host": "localhost",
420+
"port": worker_port,
421+
}
415422
# Update the list of stream writers. It's convenient that the name of the worker
416423
# type is the same as the stream to write. Iterate over the whole list in case there
417424
# is more than one.
@@ -423,10 +430,15 @@ def add_worker_roles_to_shared_config(
423430

424431
# Map of stream writer instance names to host/ports combos
425432
# For now, all stream writers need http replication ports
426-
instance_map[worker_name] = {
427-
"host": "localhost",
428-
"port": worker_port,
429-
}
433+
if os.environ.get("SYNAPSE_USE_UNIX_SOCKET", False):
434+
instance_map[worker_name] = {
435+
"path": f"/run/worker.{worker_port}",
436+
}
437+
else:
438+
instance_map[worker_name] = {
439+
"host": "localhost",
440+
"port": worker_port,
441+
}
430442

431443

432444
def merge_worker_template_configs(
@@ -718,17 +730,29 @@ def generate_worker_files(
718730
# Note that yaml cares about indentation, so care should be taken to insert lines
719731
# into files at the correct indentation below.
720732

733+
# Convenience helper for if using unix sockets instead of host:port
734+
using_unix_sockets = environ.get("SYNAPSE_USE_UNIX_SOCKET", False)
721735
# First read the original config file and extract the listeners block. Then we'll
722736
# add another listener for replication. Later we'll write out the result to the
723737
# shared config file.
724-
listeners = [
725-
{
726-
"port": MAIN_PROCESS_REPLICATION_PORT,
727-
"bind_address": MAIN_PROCESS_LOCALHOST_ADDRESS,
728-
"type": "http",
729-
"resources": [{"names": ["replication"]}],
730-
}
731-
]
738+
listeners: List[Any]
739+
if using_unix_sockets:
740+
listeners = [
741+
{
742+
"path": MAIN_PROCESS_UNIX_SOCKET_PRIVATE_PATH,
743+
"type": "http",
744+
"resources": [{"names": ["replication"]}],
745+
}
746+
]
747+
else:
748+
listeners = [
749+
{
750+
"port": MAIN_PROCESS_REPLICATION_PORT,
751+
"bind_address": MAIN_PROCESS_LOCALHOST_ADDRESS,
752+
"type": "http",
753+
"resources": [{"names": ["replication"]}],
754+
}
755+
]
732756
with open(config_path) as file_stream:
733757
original_config = yaml.safe_load(file_stream)
734758
original_listeners = original_config.get("listeners")
@@ -769,7 +793,17 @@ def generate_worker_files(
769793

770794
# A list of internal endpoints to healthcheck, starting with the main process
771795
# which exists even if no workers do.
772-
healthcheck_urls = ["http://localhost:8080/health"]
796+
# This list ends up being part of the command line to curl, (curl added support for
797+
# Unix sockets in version 7.40).
798+
if using_unix_sockets:
799+
healthcheck_urls = [
800+
f"--unix-socket {MAIN_PROCESS_UNIX_SOCKET_PUBLIC_PATH} "
801+
# The scheme and hostname from the following URL are ignored.
802+
# The only thing that matters is the path `/health`
803+
"http://localhost/health"
804+
]
805+
else:
806+
healthcheck_urls = ["http://localhost:8080/health"]
773807

774808
# Get the set of all worker types that we have configured
775809
all_worker_types_in_use = set(chain(*requested_worker_types.values()))
@@ -806,8 +840,12 @@ def generate_worker_files(
806840
# given worker_type needs to stay assigned and not be replaced.
807841
worker_config["shared_extra_conf"].update(shared_config)
808842
shared_config = worker_config["shared_extra_conf"]
809-
810-
healthcheck_urls.append("http://localhost:%d/health" % (worker_port,))
843+
if using_unix_sockets:
844+
healthcheck_urls.append(
845+
f"--unix-socket /run/worker.{worker_port} http://localhost/health"
846+
)
847+
else:
848+
healthcheck_urls.append("http://localhost:%d/health" % (worker_port,))
811849

812850
# Update the shared config with sharding-related options if necessary
813851
add_worker_roles_to_shared_config(
@@ -826,6 +864,7 @@ def generate_worker_files(
826864
"/conf/workers/{name}.yaml".format(name=worker_name),
827865
**worker_config,
828866
worker_log_config_filepath=log_config_filepath,
867+
using_unix_sockets=using_unix_sockets,
829868
)
830869

831870
# Save this worker's port number to the correct nginx upstreams
@@ -846,8 +885,13 @@ def generate_worker_files(
846885
nginx_upstream_config = ""
847886
for upstream_worker_base_name, upstream_worker_ports in nginx_upstreams.items():
848887
body = ""
849-
for port in upstream_worker_ports:
850-
body += f" server localhost:{port};\n"
888+
if using_unix_sockets:
889+
for port in upstream_worker_ports:
890+
body += f" server unix:/run/worker.{port};\n"
891+
892+
else:
893+
for port in upstream_worker_ports:
894+
body += f" server localhost:{port};\n"
851895

852896
# Add to the list of configured upstreams
853897
nginx_upstream_config += NGINX_UPSTREAM_CONFIG_BLOCK.format(
@@ -877,10 +921,15 @@ def generate_worker_files(
877921
# If there are workers, add the main process to the instance_map too.
878922
if workers_in_use:
879923
instance_map = shared_config.setdefault("instance_map", {})
880-
instance_map[MAIN_PROCESS_INSTANCE_NAME] = {
881-
"host": MAIN_PROCESS_LOCALHOST_ADDRESS,
882-
"port": MAIN_PROCESS_REPLICATION_PORT,
883-
}
924+
if using_unix_sockets:
925+
instance_map[MAIN_PROCESS_INSTANCE_NAME] = {
926+
"path": MAIN_PROCESS_UNIX_SOCKET_PRIVATE_PATH,
927+
}
928+
else:
929+
instance_map[MAIN_PROCESS_INSTANCE_NAME] = {
930+
"host": MAIN_PROCESS_LOCALHOST_ADDRESS,
931+
"port": MAIN_PROCESS_REPLICATION_PORT,
932+
}
884933

885934
# Shared homeserver config
886935
convert(
@@ -890,6 +939,7 @@ def generate_worker_files(
890939
appservice_registrations=appservice_registrations,
891940
enable_redis=workers_in_use,
892941
workers_in_use=workers_in_use,
942+
using_unix_sockets=using_unix_sockets,
893943
)
894944

895945
# Nginx config
@@ -900,6 +950,7 @@ def generate_worker_files(
900950
upstream_directives=nginx_upstream_config,
901951
tls_cert_path=os.environ.get("SYNAPSE_TLS_CERT"),
902952
tls_key_path=os.environ.get("SYNAPSE_TLS_KEY"),
953+
using_unix_sockets=using_unix_sockets,
903954
)
904955

905956
# Supervisord config
@@ -909,6 +960,7 @@ def generate_worker_files(
909960
"/etc/supervisor/supervisord.conf",
910961
main_config_path=config_path,
911962
enable_redis=workers_in_use,
963+
using_unix_sockets=using_unix_sockets,
912964
)
913965

914966
convert(

0 commit comments

Comments
 (0)