diff --git a/docs/TESTS.md b/docs/TESTS.md index 925189ab2f..40a1846cfe 100644 --- a/docs/TESTS.md +++ b/docs/TESTS.md @@ -1,6 +1,6 @@ # Running tests -From the terminal, change to the base directory of the problem then execute the tests with: +From the terminal, change to the base directory of the exercise then execute the tests with: ```bash $ mix test @@ -10,7 +10,7 @@ This will execute the test file found in the `test` subfolder -- a file ending i ## Pending tests -In the test suites, all but the first test have been skipped. +In test suites of practice exercises, all but the first test have been tagged to be skipped. Once you get a test passing, you can unskip the next one by commenting out the relevant `@tag :pending` with a `#` symbol. @@ -24,6 +24,12 @@ test "shouting" do end ``` +If you wish to run all tests at once, you can include all skipped test by using the `--include` flag on the `mix test` command: + +```bash +$ mix test --include pending +``` + Or, you can enable all the tests by commenting out the `ExUnit.configure` line in the test suite. @@ -31,6 +37,101 @@ Or, you can enable all the tests by commenting out the # ExUnit.configure exclude: :pending, trace: true ``` +## Additional Elixir testing features + +`ExUnit` and `mix test` provide quite a few methods of grouping, tagging & executing +tests and various methods of controlling test execution, much of which is summarized +below. + +### Methods for running specific tests + +Documentation: + +* [`mix test` documentation](https://hexdocs.pm/mix/Mix.Tasks.Test.html) + +#### Running tests in a specific file + +All tests in a single file may be executed via `mix test` by specifying the file: + +```bash +$ mix test test/.exs +``` + +> NOTE: `tagging` may impact which tests are actually executed using this method. + +#### Running individual tests + +Individual tests may be executed by referencing a test's line number in the file: + +```bash +$ mix test test/.exs:LINENUM +``` + +Multiple tests may be executed by giving multiple line numbers separated by `:`. + +For example, given a file with the following content with line numbers: + +```elixir +test "Test 1" do # 1 + # test implementation # 2-6 +end # 7 + # 8 +test "Test 2" do # 9 + # test implementation # 10-21 +end # 22 + # 23 +test "Test 3" do # 24 + # test implementation # 25-35 +end # 36 +``` + +The 1st and 3rd tests can be executed by: + +```bash +$ mix test test/FILE.exs:1:24 +``` + +> NOTE: When specifying tests via line numbers, `tagging` is ignored. + +#### Running groups of tests + +Tests may be grouped using `describe`: + +```elixir +describe "short test group description" do + test "test description" do + # test implementation + end + + test "another test description" do + # test implementation + end +end +``` + +All tests in a group may be executed by referencing its line number in the file, +just like referencing and executing individual tests. + +Documentation: + +* [`describe`](https://hexdocs.pm/ex_unit/ExUnit.Case.html#describe/2) + +#### Other useful `mix test` options + +* `--include` and `--exclude` - runs or doesn't run specific tests based on their `@tag`s +* `--failed` - runs only tests that failed the last time they ran +* `--max-failures` - the suite stops evaluating tests when this number of test failures + is reached +* `--seed` - seeds the random number generator used to randomize the order of tests + `--seed 0` disables randomization so the tests in a single file will always be ran + in the same order they were defined in +* `--stale` - runs only tests which reference modules that changed since the last + time tests were ran with `--stale` + +Documentation: + +* [`mix test` command-line options](https://hexdocs.pm/mix/Mix.Tasks.Test.html#module-command-line-options) + ## Typespecs and Dialyzer (DIscrepancy AnalYZer for ERlang programs) Elixir exercises include a skeleton implementation file in the `lib` diff --git a/exercises/shared/.docs/tests.md b/exercises/shared/.docs/tests.md index cbd4c56ae2..26469a7472 100644 --- a/exercises/shared/.docs/tests.md +++ b/exercises/shared/.docs/tests.md @@ -1,14 +1,21 @@ # Running tests -Execute the tests with: +From the terminal, change to the base directory of the exercise then execute the tests with: ```bash $ mix test ``` +This will execute the test file found in the `test` subfolder -- a file ending in `_test.exs` + +Documentation: + +* [`mix test` - Elixir's test execution tool](https://hexdocs.pm/mix/Mix.Tasks.Test.html) +* [`ExUnit` - Elixir's unit test library](https://hexdocs.pm/ex_unit/ExUnit.html) + ## Pending tests -In the test suites, all but the first test have been skipped. +In test suites of practice exercises, all but the first test have been tagged to be skipped. Once you get a test passing, you can unskip the next one by commenting out the relevant `@tag :pending` with a `#` symbol. @@ -32,3 +39,12 @@ Or, you can enable all the tests by commenting out the `ExUnit.configure` line i ```elixir # ExUnit.configure(exclude: :pending, trace: true) ``` + +## Useful `mix test` options + +* `test/.exs:LINENUM` - runs only a single test, the test from `.exs` whose definition is on line `LINENUM` +* `--failed` - runs only tests that failed the last time they ran +* `--max-failures` - the suite stops evaluating tests when this number of test failures +is reached +`--seed 0` - disables randomization so the tests in a single file will always be ran +in the same order they were defined in