beforeAll() executed in parallel with test case in case of timeout

[aalexgabi]

🐛 Bug Report

Test case is executed even if beforeAll did not finish.

To Reproduce

This test file:

beforeAll(async () => new Promise((resolve, reject) => {
  // Never resolves
}));

test('test case', async () => {
  console.log('should never be executed');
});

Displays this:

% npx jest t.spec.js
 FAIL  ./t.spec.js (5.415s)
  ✕ test case (8ms)

  ● test case

    Timeout - Async callback was not invoked within the 5000ms timeout specified by jes
t.setTimeout.Error: Timeout - Async callback was not invoked within the 5000ms timeout 
specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:25:45)

  console.log t.spec.js:6
    should never be executed

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        5.436s, estimated 6s
Ran all test suites matching /t.spec.js/i.

Expected behavior

Test case code should never be executed because beforeAll() did not finish. The error message should be at the beforeAll level and not at the test case level. This is very confusing as you may think that the test case itself failed when in reality it has nothing to do with the test case.

Link to repl or repo (highly encouraged)

https://repl.it/repls/KindlyAutomaticLocation

envinfo

% npx envinfo --preset jest
npx: installed 1 in 0.645s

  System:
    OS: Linux 5.3 Manjaro Linux
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  Binaries:
    Node: 13.7.0 - /usr/bin/node
    Yarn: 1.21.1 - /usr/bin/yarn
    npm: 6.13.6 - /usr/bin/npm
  npmPackages:
    jest: ^24.9.0 => 24.9.0 

[dfleury]

Still occuring with jest 25.1.0.
I'm starting a http server and waiting the server.listen() callback to be called to resolve the promise and ensure the server is able to receive requests. However, the testcase is running before the server is up and running because the runner isn't waiting beforeAll finish their job, just like @aalexgabi reported.

[dfleury]

Seems I've found a workaround here. Since I wrapped my test case and hooks with a describe function, seems that jest is now respecting the async beforeAll.

[aalexgabi]

@dfleury I do not observe the same behaviour:

% cat test/t.spec.js 
describe('main', () => {
  beforeAll(async () => new Promise((resolve, reject) => {
    // Never resolves
  }));

  test('test case', async () => {
    console.log('should never be executed');
  });
});

% npx jest test/t.spec.js 
 FAIL  test/t.spec.js (5.296s)
  main
    ✕ test case (5ms)

  ● main › test case

    Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:25:45)

  console.log test/t.spec.js:7
    should never be executed

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        5.646s, estimated 6s
Ran all test suites matching /test\/t.spec.js/i.

% npx envinfo --preset jest
npx: installed 1 in 0.727s

  System:
    OS: Linux 5.3 Manjaro Linux
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  Binaries:
    Node: 13.7.0 - /usr/bin/node
    Yarn: 1.22.0 - /usr/bin/yarn
    npm: 6.13.6 - /usr/bin/npm
  npmPackages:
    jest: ^25.1.0 => 25.1.0 

[dfleury]

I guess the problem in your example is you're never resolving the promise and the global timeout is firing because of that.
What I've catched here is similar but different since my test case was running in parallel with beforeAll even I'm respecting the 5s limit and, a variable that might be defined by beforeAll was being used before it was ready. That way, my test case failed saying that variable X wasn't defined.
In a tweet, I think the issue here is jest isn't respecting the execution order of a global beforeAll (out of any scope).

[fvbrennan]

We have been bitten by this unexpected behaviour as well, here is a simple repro case where the log output will show the improper sequencing

//Keeping default jest timeout of 5s

beforeAll(async () => {
  console.log("In beforeAll, starting sleep");
  await sleep(12);
  console.log("beforeAll finally finished sleeping");
});

describe("outerDescribe", () => {
  beforeEach(async () => {
    console.log("In beforeEach, starting sleep");
    await sleep(6);
    console.log("beforeEach finally finished sleeping");
  });
  describe("innerDescribe", () => {
    it("Does absolutely nothing useful", () => {
      console.log("Executing testcase 1");
    });
    it("Also does nothing useful", () => {
      console.log("Executing testcase 2");
    });
  });
});

function sleep(seconds) {
  return new Promise(resolve => setTimeout(resolve, seconds * 1000));
}

Output:
In beforeAll, starting sleep
In beforeEach, starting sleep
Executing testcase 1
Error: Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.
console.log app/tests/test.js:9
In beforeEach, starting sleep
beforeEach finally finished sleeping
beforeAll finally finished sleeping
Executing testcase 2
Error: Timeout - Async callback was not invoked within the 5000ms timeout specified by

[thernstig]

Same with using beforeEach().

[albert-schilling]

I have the same issue. Planned on using a deleteAllEntries() function to clean my collection in db before tests and placed the function in beforeAll(), but apparently other test functions are already adding entries to the db, while deleteAllEntries() has not yet finished.

The parallel execution of beforeAll() and afterAll() with the tests somehow defies their inherent purpose?

Anyone found a good solution to this?

[mateuspiresl]

I'm facing this issue.
Agree with @albert-schilling. There is no point in testing being run before preparation (beforeAll and beforeEach) is complete.

[eps1lon]

Am I understanding this correctly that jest is basically telling that you shouldn't share context between tests at all? For example, with puppeteer/playwright it's commonly documented to do

describe.each(["chromium", "firefox"])("%s", (browserType) => {
	let browser;
	let page;

	beforeAll(async () => {
		browser = await playwright[browserType].launch();
	});

	afterAll(async () => {
		await browser.close();
	});

	beforeEach(async () => {
		page = await browser.newPage();
	});

	afterEach(async () => {
		await page.close();
	});

  it('first test', async () => {
    //  do something with `page`
  });

  it('second test', () => {
    // do something with a new `page` that doesn't interfere with the first test
  });
});

With jest that doesn't seem possible since an afterEach which closes the page could run during one of the test because jest doesn't wait for before*/after* to finish?

It's not a blocker since I can create a new browser and page per test but then what is the point of before* and after hooks?

[wanton7]

Our company lost 4 man-days of development because of this bug. Please fix this so others don't suffer the same fate. Return proper error for this and don't run test if beforeAll etc timeouts. So they know that they need to put bigger timeout as second argument for beforeAll and afterAll etc.