Wish for local development: run 1 command to start all services instead of 3 commands

[carols10cents]

I've been thinking about this and trying some things for a while but I haven't had much luck, so I'm putting this out there in case someone else would like to try.

Here's my ideal workflow for local development:

• Check out whatever branch/PR I want to work on
• Run 1 command that starts up:
  • cargo run --bin server
  • cargo run --bin background-worker
  • npm install && npm run start:local (I switch branches that change our JavaScript dependencies a lot)
• Notify me somehow that everything is ready, or that something failed
  • desktop notification? hard to do cross-platform, maybe there's a crate for that.
  • opening a browser tab? might interrupt whatever I've switched to doing instead, would probably end up with me having a million tabs open. if failing just doesn't open a browser window, though, i probably would waste time before i go check.
  • something else??
• Have all the logs in that window, but identify which line comes from which process.
• When I CTRL-C that one command, all the running processes stop.

Bonus would be running diesel migration run before either of the cargo run commands start.

I think it would be really nice to have this optionally available to people working on the codebase; it could help us with forgetting to document how to run the codebase locally if there's a file that we add the commands to that we use regularly, rather than the contributing documentation that we let go out of date.

Things I've considered:

• Shell script, but I don't feel like I know enough to do this properly, mostly the "stop all processes when the main one is stopped" part. And the "wait for all of the commands to be 'ready'" part. Actually all the parts.
• foreman, which I'm familiar with from my Ruby days, but I don't want to add installing Ruby to our development dependencies. There are some ports of foreman, even some Rust ones, but I haven't evaluated any of those.
  • Even if we do use the Ruby foreman and make this setup optional, I've had some issues. Here's a branch where I've tried using foreman, I've hit these problems:
    • Running diesel migration run && before one of the two cargo run commands means the other one might start before the migration is done and that might cause the build to fail.
    • I'm hardcoding the ports in the Procfile because the package.json has 8888 hardcoded for where the backend runs, and I expect to go to localhost:4200 in my browser. I've made sure that there's nothing running on port 4200, but when I try to run with foreman, it always says the frontend can't start because something else is using port 4200. I have no idea how it's blocking itself with itself, and only when run from foreman.
    • Running diesel migration run && before both of the cargo run commands usually errors because running the migrations makes changes to the current schema file, and the other migration process will get into a bad state and fail.
    • Leaving out all attempts at running migrations, sometimes the two cargo run processes deadlock:

$ foreman start -f Procfile.local
13:33:57 web.1               | started with pid 61153
13:33:57 background_worker.1 | started with pid 61154
13:33:57 ui.1                | started with pid 61155
13:33:57 web.1               |     Blocking waiting for file lock on package cache
13:33:57 background_worker.1 |     Blocking waiting for file lock on package cache
13:33:58 web.1               |     Blocking waiting for file lock on package cache
13:33:58 background_worker.1 |     Blocking waiting for file lock on package cache
13:33:58 web.1               |     Blocking waiting for file lock on package cache
13:33:58 background_worker.1 |     Blocking waiting for file lock on build directory
13:33:58 web.1               |     Blocking waiting for file lock on build directory

We could build first and then do ./target/debug/server and ./target/debug/background-worker instead, but like with the diesel command, I'm not sure how to say "run this command and when that finishes run these two other commands"

• Is there some other tool that would be good for this that I haven't considered?

[smarnach]

We already have the docker-compose configuration. Docker-compose can more or less meet the requirements you list, but also adds more complexity than I like.

I'd prefer to avoid a Ruby dependency, even an optional one for development. We don't have that many developers working on the project, and we already have various different development setups (at least Linux bare metal, macOS bare metal, docker-compose). Adding further diversity ressults in too many options to reliably keep working.

The Node port of foreman is quite popular, so we may want to look into it. If there's a good Rust port, that's definitely an option as well.

To run commands before starting the foreman jobs we could simply write a shell script (or Makefile) that runs the mgirations, builds and then calls foreman.

[kzys]

I can agree Docker is complex, but having multiple simple configurations also make another kind of complexity.

[plaets]

Update: docker-compose currently does migrations, starts both the backend and the frontend, and with #3223 (merged) also starts the background worker. Logs contain service identification.
Looks like the only part of this issue that is missing, is the part with notifying the user when everything is ready or when something failed to start.

[Turbo87]

Given the above comment I'm going to close this issue as completed :)