Updates to test execution documentation (#940)

Co-authored-by: Angelika Tyborska <[email protected]>
Co-authored-by: czrpb <[email protected]>

Author: czrpb

docs/TESTS.md

@@ -1,6 +1,6 @@
 # Running tests
 
-From the terminal, change to the base directory of the problem then execute the tests with:
+From the terminal, change to the base directory of the exercise then execute the tests with:
 
 ```bash
 $ mix test
@@ -10,7 +10,7 @@ This will execute the test file found in the `test` subfolder -- a file ending i
 
 ## Pending tests
 
-In the test suites, all but the first test have been skipped.
+In test suites of practice exercises, all but the first test have been tagged to be skipped.
 
 Once you get a test passing, you can unskip the next one by
 commenting out the relevant `@tag :pending` with a `#` symbol.
@@ -24,13 +24,114 @@ test "shouting" do
 end
 ```
 
+If you wish to run all tests at once, you can include all skipped test by using the `--include` flag on the `mix test` command:
+
+```bash
+$ mix test --include pending
+```
+
 Or, you can enable all the tests by commenting out the
 `ExUnit.configure` line in the test suite.
 
 ```elixir
 # ExUnit.configure exclude: :pending, trace: true
 ```
 
+## Additional Elixir testing features
+
+`ExUnit` and `mix test` provide quite a few methods of grouping, tagging & executing
+tests and various methods of controlling test execution, much of which is summarized
+below.
+
+### Methods for running specific tests
+
+Documentation:
+
+* [`mix test` documentation](https://hexdocs.pm/mix/Mix.Tasks.Test.html)
+
+#### Running tests in a specific file
+
+All tests in a single file may be executed via `mix test` by specifying the file:
+
+```bash
+$ mix test test/<FILE>.exs
+```
+
+> NOTE: `tagging` may impact which tests are actually executed using this method.
+
+#### Running individual tests
+
+Individual tests may be executed by referencing a test's line number in the file:
+
+```bash
+$ mix test test/<FILE>.exs:LINENUM
+```
+
+Multiple tests may be executed by giving multiple line numbers separated by `:`.
+
+For example, given a file with the following content with line numbers:
+
+```elixir
+test "Test 1" do           # 1
+  # test implementation    # 2-6
+end                        # 7
+                           # 8
+test "Test 2" do           # 9
+  # test implementation    # 10-21
+end                        # 22
+                           # 23
+test "Test 3" do           # 24
+  # test implementation    # 25-35
+end                        # 36
+```
+
+The 1st and 3rd tests can be executed by:
+
+```bash
+$ mix test test/FILE.exs:1:24
+```
+
+> NOTE: When specifying tests via line numbers, `tagging` is ignored.
+
+#### Running groups of tests
+
+Tests may be grouped using `describe`:
+
+```elixir
+describe "short test group description" do
+  test "test description" do
+    # test implementation
+  end
+
+  test "another test description" do
+    # test implementation
+  end
+end
+```
+
+All tests in a group may be executed by referencing its line number in the file,
+just like referencing and executing individual tests.
+
+Documentation:
+
+* [`describe`](https://hexdocs.pm/ex_unit/ExUnit.Case.html#describe/2)
+
+#### Other useful `mix test` options
+
+* `--include` and `--exclude` - runs or doesn't run specific tests based on their `@tag`s
+* `--failed` - runs only tests that failed the last time they ran
+* `--max-failures` - the suite stops evaluating tests when this number of test failures
+  is reached
+* `--seed` - seeds the random number generator used to randomize the order of tests
+  `--seed 0` disables randomization so the tests in a single file will always be ran
+  in the same order they were defined in
+* `--stale` - runs only tests which reference modules that changed since the last
+  time tests were ran with `--stale`
+
+Documentation:
+
+* [`mix test` command-line options](https://hexdocs.pm/mix/Mix.Tasks.Test.html#module-command-line-options)
+
 ## Typespecs and Dialyzer (DIscrepancy AnalYZer for ERlang programs)
 
 Elixir exercises include a skeleton implementation file in the `lib`

exercises/shared/.docs/tests.md

@@ -1,14 +1,21 @@
 # Running tests
 
-Execute the tests with:
+From the terminal, change to the base directory of the exercise then execute the tests with:
 
 ```bash
 $ mix test
 ```
 
+This will execute the test file found in the `test` subfolder -- a file ending in `_test.exs`
+
+Documentation:
+
+* [`mix test` - Elixir's test execution tool](https://hexdocs.pm/mix/Mix.Tasks.Test.html)
+* [`ExUnit` - Elixir's unit test library](https://hexdocs.pm/ex_unit/ExUnit.html)
+
 ## Pending tests
 
-In the test suites, all but the first test have been skipped.
+In test suites of practice exercises, all but the first test have been tagged to be skipped.
 
 Once you get a test passing, you can unskip the next one by commenting out the relevant `@tag :pending` with a `#` symbol.
 
@@ -32,3 +39,12 @@ Or, you can enable all the tests by commenting out the `ExUnit.configure` line i
 ```elixir
 # ExUnit.configure(exclude: :pending, trace: true)
 ```
+
+## Useful `mix test` options
+
+* `test/<FILE>.exs:LINENUM` - runs only a single test, the test from `<FILE>.exs` whose definition is on line `LINENUM`
+* `--failed` - runs only tests that failed the last time they ran
+* `--max-failures` - the suite stops evaluating tests when this number of test failures
+is reached
+`--seed 0` - disables randomization so the tests in a single file will always be ran
+in the same order they were defined in