Expand and improve workers documentation and example configuration

[ShadowJonathan]

A general call;

• Add more documentation surrounding workers (just in general)
• Add example configuration file (a-la sample_config.yaml) for each worker type, or for all workers in one
• Add docker-compose file that includes ready-to-work example for synapse workers

Imo: Currently i think workers are very hard to set up or figure out on its own, because knowledge of it is mostly limited to large setups, and thus that knowledge is never reflected in documentation, or workers seen as an asset to be documented with the same priority.

[clokep]

Add more documentation surrounding workers (just in general)

Is there specific information that you think is missing? the workers docs were recently expanded to include a bit more information.

Add example configuration file (a-la sample_config.yaml) for each worker type, or for all workers in one

I think this is pretty much #8557.

Add docker-compose file that includes ready-to-work example for synapse workers

The current docker-compose stuff is under contrib, so I unfortunately this will probably need to be provided via a contribution. 😄

[ShadowJonathan]

@clokep why was info-needed added here, i thought i was pretty clear; improve workers documentation and add samples, for the first one there's already an issue (good, cuz then i can track that via here), and for the second one i havent seen a thing (with issue or PR), or should that issue be added to another repo?

[clokep]

@ShadowJonathan The bit that is unclear to me is:

Add more documentation surrounding workers (just in general)

Is there specific information that you think is missing? the workers docs were recently expanded to include a bit more information.

What information do you feel is missing?

[ShadowJonathan]

Oh, that part is a general call for a look at the documentation, but i'm not holding it for a requirement (thus the absence of checkmark), its generally just "hey, please give a look at the documentation for workers and judge if everything you know about setting up and configuring workers is in there" request, should i remove it to avoid confusion?

[richvdh]

we certainly need better documentation for how to run workers under docker

[TheDiscordian]

I was asked to drop my opinion in here while we were discussing this issue in chat:

There's a lot of info on types of workers, and little on how to redirect queries from a reverse proxy to the workers effectively. I know it sounds simple to say "just run a worker per CPU", but I didn't find my experience to feel quite so clear-cut.

Put simply, an example showing how to run a server with 4+ cores, using Nginx as a reverse proxy, with an appropriate worker setup / Nginx config would go a long way I think.

[ShadowJonathan]

Extra thoughts;

• docker-compose sample
• full setup sample config