Too many tests for cater-waiter

[rcardenes]

While testing Cater Waiter I noticed that the tests were timing out. After discussing it with @BethanyG she figured out that there are way too many tests, which causes the system to time out while trying to run them all, particularly when they fail.

[BethanyG]

@rcardenes -- I am assigning this to myself, since I can't assign it directly to you (yet). Many thanks for the testing and the assist!

[BethanyG]

I will tag the PR as medium or large work (depending) when you submit it, since this will take some time and effort to thin out, and therefore deserves an added rep bump. 😄

[Ou42]

Warning: NOOB Q's -- but ... Must the tests be thinned or would it be possible to speed up the tests?
( guessing here, but is it possible to keep things in mem for subsequent tests? and/or could the tests catch that "pass" wasn't changed and then not run subsequent tests? )

[rcardenes]

I believe the problem is not the tests themselves, which in principle run fast enough: as you start passing tests, eventually your results come up. The problem would be the huge amount of output that they produce that have then to be made into a report and presented to the user.

[BethanyG]

@Ou42 -- great question -- and yes -- if it were test speed we might want to consider some strategies for making data loading or execution faster. But all 120+ tests for this exercise run in well under 6s -- even with loading data.

As @rcardenes mentions, the particular issue here isn't that the tests or the test runner is bogging down and timing out -- it is that the test report for all of the failures for an empty file is so verbose, it is timing out / screwing up the UI component meant to display the test results on the website (and no doubt stressing out anyone using TDD with the CLI). 😱

The test writer (me) was lazy and included very large equivalency classes. Those equivalency classes need to be slimmed down, as they are not necessary -- we don't need 25 variations on every test if 5 will fulfill the requirement just as well. 😅

[Ou42]

@rcardenes & @BethanyG - thank you both for your answers. Very informative! ( & also thank you for your efforts! )