Should unit tests spawn new processes instead of threads?

[vlovich]

Imagine a (contrived) piece of code like this:

lazy_static!(Q: Mutex<Vec<u32>> = Mutex::new(Vec::new()));

#[test]
fn test_a() {
   {
     Q.lock().unwrap().push(5);
   }
   assert_eq!(Q.lock().unwrap().len(), 1);
}

fn test_b() {
   {
     Q.lock().unwrap().clear();
   }
   assert_eq!(Q.lock().unwrap().len(), 0);

   {
     let q = Q.lock().unwrap();
     q.push(5);
     q.push(6);
   }
   assert_eq!(Q.lock().unwrap().len(), 2);
}

This is broken by default because cargo test will by default spawn multiple threads for the tests so the global state of Q. The user would have to know to pass --test-threads=1 or the test author would have to create a global mutex to synchronize each test on. This seems like a lot of unnecessary boilerplate, not to mention an unnecessary pain point for new developers to discover when they can't figure out why tests are producing weird results.

There also doesn't seem to be a solution at the moment that would enable you to achieve the benefits of running tests in parallel at all for shared state between tests. The global mutex approach effectively reduces the test code to implicitly running with --test-threads=1. Using separate processes by default (or at least providing a decorator to do it easily) would solve the vast majority of such problems without requiring authors to significantly restructure their code.

FWIW, other test frameworks go with the process sharding approach instead of threads.

[karbfg10k]

I'd really appreciate this at some point. I'm using Rust to functionally verify my companies C code base. Our code is automotive grade(runs on ECU's) so it uses lots of global's and massive functions with significant internal state (the code is auto generated from a model). Being able to spawn tests as processes would allow tests to run in parallel which would be a huge win.

[jorickert]

rusty-fork can be used to work around this issue

[Mark-Simulacrum]

cc #32512

[jrose-signal]

Poking this old issue because having an option to force this, even if it isn't the default, would be very beneficial for running a test suite under miri, which tracks all memory allocated over the lifetime of a program.

[jsenkpiel-godaddy]

It would be nice to be able to denote unit tests to have this as their behavior.

[vlovich]

I looked at rusty-fork just now and it seems incompatible with rstest which provides functionality to parameterize tests more easily.

[zopsicle]

Running tests in separate processes would also allow for #[should_abort] analogous to #[should_panic] which can be useful for testing code that should abort. (Or some generalized way to assert a certain termination status.)

[tianyicui-tsy]

it seems rusty-fork isn't compatible with tests that use tokio (i.e. #[tokio-test] instead of #[test]). Would be great if this is done.