Simplify t.throws()

[novemberborn]

This is my proposal for resolving #661. In short I want to stop deferring to https://nodejs.org/api/assert.html#assert_assert_throws_block_error_message and remove various parameter overloads that are currently supported by the assertion.

Here's what I'd like to support:

t.throws(fn) // Throws an Error (or subclass thereof)
t.throws(fn, SyntaxError) // Throws a SyntaxError (or subclass thereof)
t.throws(fn, 'string') // Throws an Error (or subclass thereof), whose .message === 'string'
t.throws(fn, /regexp/) // Throws an Error (or subclass thereof), whose /regexp/.test(err.message) === true

If you need to test the errors class and message you should use the t.throws return value:

const err = t.throws(fn, SyntaxError)
t.true(err.message === 'expecting integer')

Passing anything other than undefined, functions, strings or regular expressions as the second argument results in an TypeError from t.throws() itself.

The first argument is allowed to be a promise or observable, as is any return value from the fn call. Of course this makes t.throws asynchronous, so users may need to do:

const err = await t.throws(fn)

---

Questions:

1. Does t.throws(fn, Constructor) require the thrown value to be an Error? For example:

   t.throws(() => { throw 'foo' }, String) // Is this allowed?

2. Should t.throws(fn, 'string') and t.throws(fn, /regexp/) be supported at all, or would it be preferable to dereference the error and then use another assertion?

   const err = t.throws(fn)
   t.true(err.message === 'expecting integer')

3. Is there a need for t.throws(fn, SyntaxError, 'string') and t.throws(fn, SyntaxError, /regexp/)

4. Does anybody have experience with / examples of asserting Error instances across contexts?

[vadimdemedes]

I absolutely support your proposal!

Answers to your questions:

1. I don't think it should be allowed. ESLint's docs on no-throw-literal rule explain it best:

It is considered good practice to only throw the Error object itself or an object using the Error object as base objects for user-defined exceptions. The fundamental benefit of Error objects is that they automatically keep track of where they were built and originated.

Services like BugSnag, Sentry, Rollbar, etc will provide zero information, if code throws strings, instead of Error objects.

2. I think they should be supported, with behavior exactly like you outlined in your first code example:

t.throws(fn, 'string') // Throws an Error (or subclass thereof), whose .message === 'string'
t.throws(fn, /regexp/) // Throws an Error (or subclass thereof), whose /regexp/.test(err.message) === true

I use this way constantly and find it very convenient.

3. I think this would also be a nice thing to have, but not insisting on this one. Let's hear from others whether they use this way or not.

4. Didn't completely understand the question, mind providing a quick example?

[fearphage]

The current implementation seems to almost match your proposal. t.throws(fn, type, message) doesn't work (0.16.0). I can't tell if this is a regression or a mix up in the documentation. I went back 10 or so commits and never saw where this worked. There are no tests for this format either.

I was coming here to file that bug report and submit the PR, but I won't if you're planning to remove it altogether.

[novemberborn]

Conclusions so far:

Does t.throws(fn, Constructor) require the thrown value to be an Error?

Yes.

Should t.throws(fn, 'string') and t.throws(fn, /regexp/) be supported at all, or would it be preferable to dereference the error and then use another assertion?

We'll support this interface.

Is there a need for t.throws(fn, SyntaxError, 'string') and t.throws(fn, SyntaxError, /regexp/)

Leaning towards not supporting this.

---

@vdemedes

Does anybody have experience with / examples of asserting Error instances across contexts?

Didn't completely understand the question, mind providing a quick example?

const err1 = new Error()
const err2 = vm.runInNewContext('new Error()')
err1 instanceof Error // true
err2 instanceof Error // false

---

@fearphage yea AVA just uses Node.js' API, and it gets messy fast. Hence this proposal.

[fearphage]

Is there a need for t.throws(fn, SyntaxError, 'string') and t.throws(fn, SyntaxError, /regexp/)

Leaning towards not supporting this.

Is it difficult to code or reason about? Just a preference? I'm just curious.

If we're target an arity of 2, what about this as an option?

t.throws(fn, error => {
  // insert error assertions
});

I like the encapsulation of the assertions related to the thrown error.

[novemberborn]

Is it difficult to code or reason about? Just a preference? I'm just curious.

The assertion becomes overly complex again, which is what we're trying to move away from.

If we're target an arity of 2, what about this as an option?

t.throws(fn, error => {
  // insert error assertions
});

I like the encapsulation of the assertions related to the thrown error.

That'll be hard to distinguish from t.throws(fn, Constructor).

[jfmengels]

I like the proposal :)

1. Does t.throws(fn, Constructor) require the thrown value to be an Error?

Yes. That is a best practice, or rather, using a string is a bad practice, and we should help the user remove that bad practice from their code IMO.
We should also (and I don't remember whether this is already something that we're doing) fail and show an error when the thrown error is a string and not an Error.
It's a bit different if an underlying library throws a string, but even then it should be simple for the user to get around that and do a normal try/catch instead.

1. Should t.throws(fn, 'string') and t.throws(fn, /regexp/) be supported at all, or would it be preferable to dereference the error and then use another assertion?

That is common enough to keep IMO.

1. Is there a need for t.throws(fn, SyntaxError, 'string') and t.throws(fn, SyntaxError, /regexp/)

I don't custom Error types that much. I think it's rare enough and simple enough to get around it

const err = t.throws(fn, 'string');
t.true(err instanceof SyntaxError);

1. Does anybody have experience with / examples of asserting Error instances across contexts?

No :/

---

From what I see, you're proposing the removal of the message too? That would make it inconsistent with the other APIs. Not sure I agree here.

[novemberborn]

We should also (and I don't remember whether this is already something that we're doing) fail and show an error when the thrown error is a string and not an Error.

Yes that's implied.

From what I see, you're proposing the removal of the message too? That would make it inconsistent with the other APIs. Not sure I agree here.

Oh no! Forgot to add that…

I think that rules out syntax like t.throws(fn, SyntaxError, 'string'), since that collides with t.throws(fn, SyntaxError, 'message'). Padding it is just annoying (t.throws(fn, SyntaxError, undefined, 'message'). Though you would need to pad t.throws(fn, undefined, 'message'). I think that's OK though.

[sindresorhus]

Alternative proposal.

Make it t.throws(fn, matchers, [message]);, where matchers is an object of different matchers that all has to match. Matchers can be string or regex.

This could easily be an addition to supporting string/regex/constructor in the second argument, for backwards compatibility.

For example:

t.throws(() => foo(), {
    constructor: RangeError,
    message: /unicorn pasta is/
});

Compared to the current way:

const err = t.throws(() => foo(), /unicorn pasta is/);
t.true(err instanceof RangeError);

Possible matchers constructor, message. Could also consider supporting name for when you don't have easy access to the constructor function. Maybe also stack if you want to ensure something is part of it, though I don't think I've ever needed that.

Benefits of this is that it makes everything explicit. No more wondering for users what t.throws(() => foo(), 'Unicorn') matches against - Is it the name, type, or message. There's also benefit in being able to specify all the constraints directly instead of in separate calls later, like potential for a better diff.

---

We should also have much better validation logic to prevent mistakes. For example, using is-error-constructor on the constructor matcher.

[novemberborn]

I like it!