t.throws shows `undefined undefined undefined` when a string is thrown

[SamVerschueren]

Description

When using t.throws() with a promise that rejects with a string instead of an Error object, the assertion fails.

import test from 'ava';

async function fn(): Promise<any> {
    return Promise.reject('foo');
}

test(async t => {
    t.throws(fn(), 'foo');
});

When using Promise.reject(new Error('foo'));, it works as expected.

It's due to these lines. I have no idea why it overrides the err reference with a function. I looked into core-assert and it doesn't seem like it works with passing in a function as the expected parameter. But might be missing something here.

Environment

Node.js v4.2.1
darwin 15.3.0

[novemberborn]

@SamVerschueren that is expected. With 0.13.0 t.throws() only tests against errors, before it would try and cast other types to errors, which was wrong.

Note that your examples are different. In the first you reject with a string, in the second with an error.

[SamVerschueren]

Oh ok, I thought that it was still working with 0.13.x. My mistake then. Good to know :).

[sindresorhus]

I think our output could be better than just undefined undefined undefined though.

[SamVerschueren]

You're right!

[SamVerschueren]

Just to clear things up a bit regarding this issue. Should AVA handle the case where a dev throws something else then an Error (string, boolean, number, ...)? Or should it show an error that he should throw an error instead? Like implemented in my 2 PRs?

[jamestalmage]

@SamVerschueren
Sorry I haven't gotten to review your PR's yet. There are two closed issues that discuss the expected behaviors of t.throws.

#468 (comment)

It should return the thrown Error (if it's synchronous). Or a Promise for the rejection reason if an async object is returned (be it Observable or Promise). In other words, t.throws should turn a rejected promise into a resolved promise for the rejection reason. This allows you to await the rejection reason for additional assertions (if you are attaching additional properties).

#493 (comment)

The assertion should fail if non-Errors are used. This includes both the sync and async versions. The message should be a very helpful message that explains "You threw a non-Error, which is bad form". In other words, we should minimize confusion for developers who intentionally threw a non-Error and are trying to make assertions against it, that that is not supported behavior. If users want to assert against non-Errors, they need to catch them themselves.

---

This is complicated enough that we should pretty thoroughly document it. Perhaps it's enough to justify a recipe: "Properly Handling Errors in AVA" that includes detailed explanations on the pitfalls of non-Errors.

---

There is a limited value to grabbing non-Error's, and that is specifically when you are writing a framework or tool that executes user supplied callbacks. In that case it's often beneficial to write a few assertions to define how your code responds when your users code throws non-Errors. Since that's the rarer corner case, you should be responsible for jumping through the extra hoops to assert that behavior (by doing the more verbose thing and wrapping your tests with try/catch, etc). Again - there is probably room for a recipe here.

[novemberborn]

Node's assert.throws is rather overloaded isn't it? Maybe we could simplify the current implementation, especially given the changes from #493:

t.throws(fn) // Throws an Error (or subclass thereof)
t.throws(fn, SyntaxError) // Throws a SyntaxError (or subclass thereof)
t.throws(fn, 'string') // Throws an Error (or subclass thereof), whose .message === 'string'
t.throws(fn, /regexp/) // Throws an Error (or subclass thereof), whose /regexp/test(err.message) === true

We remove support for validation functions, preferring you use the t.throws() return value instead:

const err = t.throws(fn, TypeError)
t.true(err.message === 'expecting integer')

I'd still be in favor of the string and regular expression expectation shorthand if you're expecting an error but are not too fussed about its inheritance. Not opposed to removing that too and requiring users to run their assertion on the return value.

(Also I haven't verified whether Node's assert.throws() can deal with the subclassing)

While there's a strong convention for throwing proper errors we should still provide an assertion that can capture non-error exceptions. E.g.:

const excp = t.throwsAny(fn)
t.true(excp instanceof Foo)

[jamestalmage]

Is throwsAny really that useful? When would you ever use it?

[iamstarkov]

while upgrading ava from 0.12 to 0.13 messages like

  1. fs › should reject on empty input
  undefined undefined undefined

are not really helpful

[iamstarkov]

if i'm in favor of pure string rejection reasons, what should i do?