|
| 1 | +Come out and Clar |
| 2 | +================= |
| 3 | + |
| 4 | +In Catalan, "clar" means clear, easy to perceive. Using clar will make it |
| 5 | +easy to test and make clear the quality of your code. |
| 6 | + |
| 7 | +> _Historical note_ |
| 8 | +> |
| 9 | +> Originally the clar project was named "clay" because the word "test" has its |
| 10 | +> roots in the latin word *"testum"*, meaning "earthen pot", and *"testa"*, |
| 11 | +> meaning "piece of burned clay"? |
| 12 | +> |
| 13 | +> This is because historically, testing implied melting metal in a pot to |
| 14 | +> check its quality. Clay is what tests are made of. |
| 15 | +
|
| 16 | +## Quick Usage Overview |
| 17 | + |
| 18 | +Clar is a minimal C unit testing framework. It's been written to replace the |
| 19 | +old framework in [libgit2][libgit2], but it's both very versatile and |
| 20 | +straightforward to use. |
| 21 | + |
| 22 | +Can you count to funk? |
| 23 | + |
| 24 | +- **Zero: Initialize test directory** |
| 25 | + |
| 26 | + ~~~~ sh |
| 27 | + $ mkdir tests |
| 28 | + $ cp -r $CLAR_ROOT/clar* tests |
| 29 | + $ cp $CLAR_ROOT/test/clar_test.h tests |
| 30 | + $ cp $CLAR_ROOT/test/main.c.sample tests/main.c |
| 31 | + ~~~~ |
| 32 | + |
| 33 | +- **One: Write some tests** |
| 34 | + |
| 35 | + File: tests/adding.c: |
| 36 | + |
| 37 | + ~~~~ c |
| 38 | + /* adding.c for the "Adding" suite */ |
| 39 | + #include "clar.h" |
| 40 | + |
| 41 | + static int *answer; |
| 42 | + |
| 43 | + void test_adding__initialize(void) |
| 44 | + { |
| 45 | + answer = malloc(sizeof(int)); |
| 46 | + cl_assert_(answer != NULL, "No memory left?"); |
| 47 | + *answer = 42; |
| 48 | + } |
| 49 | + |
| 50 | + void test_adding__cleanup(void) |
| 51 | + { |
| 52 | + free(answer); |
| 53 | + } |
| 54 | + |
| 55 | + void test_adding__make_sure_math_still_works(void) |
| 56 | + { |
| 57 | + cl_assert_(5 > 3, "Five should probably be greater than three"); |
| 58 | + cl_assert_(-5 < 2, "Negative numbers are small, I think"); |
| 59 | + cl_assert_(*answer == 42, "The universe is doing OK. And the initializer too."); |
| 60 | + } |
| 61 | + ~~~~~ |
| 62 | + |
| 63 | +- **Two: Build the test executable** |
| 64 | + |
| 65 | + ~~~~ sh |
| 66 | + $ cd tests |
| 67 | + $ $CLAR_PATH/generate.py . |
| 68 | + Written `clar.suite` (1 suites) |
| 69 | + $ gcc -I. clar.c main.c adding.c -o testit |
| 70 | + ~~~~ |
| 71 | + |
| 72 | +- **Funk: Funk it.** |
| 73 | + |
| 74 | + ~~~~ sh |
| 75 | + $ ./testit |
| 76 | + ~~~~ |
| 77 | + |
| 78 | +## The Clar Test Suite |
| 79 | + |
| 80 | +Writing a test suite is pretty straightforward. Each test suite is a `*.c` |
| 81 | +file with a descriptive name: this encourages modularity. |
| 82 | + |
| 83 | +Each test suite has optional initialize and cleanup methods. These methods |
| 84 | +will be called before and after running **each** test in the suite, even if |
| 85 | +such test fails. As a rule of thumb, if a test needs a different initializer |
| 86 | +or cleanup method than another test in the same module, that means it |
| 87 | +doesn't belong in that module. Keep that in mind when grouping tests |
| 88 | +together. |
| 89 | +
|
| 90 | +The `initialize` and `cleanup` methods have the following syntax, with |
| 91 | +`suitename` being the current suite name, e.g. `adding` for the `adding.c` |
| 92 | +suite. |
| 93 | +
|
| 94 | +~~~~ c |
| 95 | +void test_suitename__initialize(void) |
| 96 | +{ |
| 97 | + /* init */ |
| 98 | +} |
| 99 | +
|
| 100 | +void test_suitename__cleanup(void) |
| 101 | +{ |
| 102 | + /* cleanup */ |
| 103 | +} |
| 104 | +~~~~ |
| 105 | +
|
| 106 | +These methods are encouraged to use static, global variables to store the state |
| 107 | +that will be used by all tests inside the suite. |
| 108 | +
|
| 109 | +~~~~ c |
| 110 | +static git_repository *_repository; |
| 111 | +
|
| 112 | +void test_status__initialize(void) |
| 113 | +{ |
| 114 | + create_tmp_repo(STATUS_REPO); |
| 115 | + git_repository_open(_repository, STATUS_REPO); |
| 116 | +} |
| 117 | +
|
| 118 | +void test_status__cleanup(void) |
| 119 | +{ |
| 120 | + git_repository_close(_repository); |
| 121 | + git_path_rm(STATUS_REPO); |
| 122 | +} |
| 123 | +
|
| 124 | +void test_status__simple_test(void) |
| 125 | +{ |
| 126 | + /* do something with _repository */ |
| 127 | +} |
| 128 | +~~~~ |
| 129 | +
|
| 130 | +Writing the actual tests is just as straightforward. Tests have the |
| 131 | +`void test_suitename__test_name(void)` signature, and they should **not** |
| 132 | +be static. Clar will automatically detect and list them. |
| 133 | +
|
| 134 | +Tests are run as they appear on their original suites: they have no return |
| 135 | +value. A test is considered "passed" if it doesn't raise any errors. Check |
| 136 | +the "Clar API" section to see the various helper functions to check and |
| 137 | +raise errors during test execution. |
| 138 | + |
| 139 | +__Caution:__ If you use assertions inside of `test_suitename__initialize`, |
| 140 | +make sure that you do not rely on `__initialize` being completely run |
| 141 | +inside your `test_suitename__cleanup` function. Otherwise you might |
| 142 | +encounter ressource cleanup twice. |
| 143 | + |
| 144 | +## How does Clar work? |
| 145 | + |
| 146 | +To use Clar: |
| 147 | + |
| 148 | +1. copy the Clar boilerplate to your test directory |
| 149 | +2. copy (and probably modify) the sample `main.c` (from |
| 150 | + `$CLAR_PATH/test/main.c.sample`) |
| 151 | +3. run the Clar mixer (a.k.a. `generate.py`) to scan your test directory and |
| 152 | + write out the test suite metadata. |
| 153 | +4. compile your test files and the Clar boilerplate into a single test |
| 154 | + executable |
| 155 | +5. run the executable to test! |
| 156 | + |
| 157 | +The Clar boilerplate gives you a set of useful test assertions and features |
| 158 | +(like accessing or making sandbox copies of fixture data). It consists of |
| 159 | +the `clar.c` and `clar.h` files, plus the code in the `clar/` subdirectory. |
| 160 | +You should not need to edit these files. |
| 161 | + |
| 162 | +The sample `main.c` (i.e. `$CLAR_PATH/test/main.c.sample`) file invokes |
| 163 | +`clar_test(argc, argv)` to run the tests. Usually, you will edit this file |
| 164 | +to perform any framework specific initialization and teardown that you need. |
| 165 | + |
| 166 | +The Clar mixer (`generate.py`) recursively scans your test directory for |
| 167 | +any `.c` files, parses them, and writes the `clar.suite` file with all of |
| 168 | +the metadata about your tests. When you build, the `clar.suite` file is |
| 169 | +included into `clar.c`. |
| 170 | + |
| 171 | +The mixer can be run with **Python 2.5, 2.6, 2.7, 3.0, 3.1, 3.2 and PyPy 1.6**. |
| 172 | + |
| 173 | +Commandline usage of the mixer is as follows: |
| 174 | + |
| 175 | + $ ./generate.py . |
| 176 | + |
| 177 | +Where `.` is the folder where all the test suites can be found. The mixer |
| 178 | +will automatically locate all the relevant source files and build the |
| 179 | +testing metadata. The metadata will be written to `clar.suite`, in the same |
| 180 | +folder as all the test suites. This file is included by `clar.c` and so |
| 181 | +must be accessible via `#include` when building the test executable. |
| 182 | + |
| 183 | + $ gcc -I. clar.c main.c suite1.c test2.c -o run_tests |
| 184 | + |
| 185 | +**Note that the Clar mixer only needs to be ran when adding new tests to a |
| 186 | +suite, in order to regenerate the metadata**. As a result, the `clar.suite` |
| 187 | +file can be checked into version control if you wish to be able to build |
| 188 | +your test suite without having to re-run the mixer. |
| 189 | + |
| 190 | +This is handy when e.g. generating tests in a local computer, and then |
| 191 | +building and testing them on an embedded device or a platform where Python |
| 192 | +is not available. |
| 193 | + |
| 194 | +### Fixtures |
| 195 | + |
| 196 | +Clar can create sandboxed fixtures for you to use in your test. You'll need to compile *clar.c* with an additional `CFLAG`, `-DCLAR_FIXTURE_PATH`. This should be an absolute path to your fixtures directory. |
| 197 | +
|
| 198 | +Once that's done, you can use the fixture API as defined below. |
| 199 | + |
| 200 | +## The Clar API |
| 201 | + |
| 202 | +Clar makes the following methods available from all functions in a test |
| 203 | +suite. |
| 204 | + |
| 205 | +- `cl_must_pass(call)`, `cl_must_pass_(call, message)`: Verify that the given |
| 206 | + function call passes, in the POSIX sense (returns a value greater or equal |
| 207 | + to 0). |
| 208 | + |
| 209 | +- `cl_must_fail(call)`, `cl_must_fail_(call, message)`: Verify that the given |
| 210 | + function call fails, in the POSIX sense (returns a value less than 0). |
| 211 | + |
| 212 | +- `cl_assert(expr)`, `cl_assert_(expr, message)`: Verify that `expr` is true. |
| 213 | + |
| 214 | +- `cl_check_pass(call)`, `cl_check_pass_(call, message)`: Verify that the |
| 215 | + given function call passes, in the POSIX sense (returns a value greater or |
| 216 | + equal to 0). If the function call doesn't succeed, a test failure will be |
| 217 | + logged but the test's execution will continue. |
| 218 | + |
| 219 | +- `cl_check_fail(call)`, `cl_check_fail_(call, message)`: Verify that the |
| 220 | + given function call fails, in the POSIX sense (returns a value less than |
| 221 | + 0). If the function call doesn't fail, a test failure will be logged but |
| 222 | + the test's execution will continue. |
| 223 | + |
| 224 | +- `cl_check(expr)`: Verify that `expr` is true. If `expr` is not |
| 225 | + true, a test failure will be logged but the test's execution will continue. |
| 226 | +
|
| 227 | +- `cl_fail(message)`: Fail the current test with the given message. |
| 228 | +
|
| 229 | +- `cl_warning(message)`: Issue a warning. This warning will be |
| 230 | + logged as a test failure but the test's execution will continue. |
| 231 | + |
| 232 | +- `cl_set_cleanup(void (*cleanup)(void *), void *opaque)`: Set the cleanup |
| 233 | + method for a single test. This method will be called with `opaque` as its |
| 234 | + argument before the test returns (even if the test has failed). |
| 235 | + If a global cleanup method is also available, the local cleanup will be |
| 236 | + called first, and then the global. |
| 237 | + |
| 238 | +- `cl_assert_equal_i(int,int)`: Verify that two integer values are equal. |
| 239 | + The advantage of this over a simple `cl_assert` is that it will format |
| 240 | + a much nicer error report if the values are not equal. |
| 241 | + |
| 242 | +- `cl_assert_equal_s(const char *,const char *)`: Verify that two strings |
| 243 | + are equal. The expected value can also be NULL and this will correctly |
| 244 | + test for that. |
| 245 | + |
| 246 | +- `cl_fixture_sandbox(const char *)`: Sets up a sandbox for a fixture |
| 247 | + so that you can mutate the file directly. |
| 248 | + |
| 249 | +- `cl_fixture_cleanup(const char *)`: Tears down the previous fixture |
| 250 | + sandbox. |
| 251 | + |
| 252 | +- `cl_fixture(const char *)`: Gets the full path to a fixture file. |
| 253 | + |
| 254 | +Please do note that these methods are *always* available whilst running a |
| 255 | +test, even when calling auxiliary/static functions inside the same file. |
| 256 | + |
| 257 | +It's strongly encouraged to perform test assertions in auxiliary methods, |
| 258 | +instead of returning error values. This is considered good Clar style. |
| 259 | +
|
| 260 | +Style Example: |
| 261 | +
|
| 262 | +~~~~ c |
| 263 | +/* |
| 264 | + * Bad style: auxiliary functions return an error code |
| 265 | + */ |
| 266 | +
|
| 267 | +static int check_string(const char *str) |
| 268 | +{ |
| 269 | + const char *aux = process_string(str); |
| 270 | +
|
| 271 | + if (aux == NULL) |
| 272 | + return -1; |
| 273 | +
|
| 274 | + return strcmp(my_function(aux), str) == 0 ? 0 : -1; |
| 275 | +} |
| 276 | +
|
| 277 | +void test_example__a_test_with_auxiliary_methods(void) |
| 278 | +{ |
| 279 | + cl_must_pass_( |
| 280 | + check_string("foo"), |
| 281 | + "String differs after processing" |
| 282 | + ); |
| 283 | +
|
| 284 | + cl_must_pass_( |
| 285 | + check_string("bar"), |
| 286 | + "String differs after processing" |
| 287 | + ); |
| 288 | +} |
| 289 | +~~~~ |
| 290 | +
|
| 291 | +~~~~ c |
| 292 | +/* |
| 293 | + * Good style: auxiliary functions perform assertions |
| 294 | + */ |
| 295 | +
|
| 296 | +static void check_string(const char *str) |
| 297 | +{ |
| 298 | + const char *aux = process_string(str); |
| 299 | +
|
| 300 | + cl_assert_( |
| 301 | + aux != NULL, |
| 302 | + "String processing failed" |
| 303 | + ); |
| 304 | +
|
| 305 | + cl_assert_( |
| 306 | + strcmp(my_function(aux), str) == 0, |
| 307 | + "String differs after processing" |
| 308 | + ); |
| 309 | +} |
| 310 | +
|
| 311 | +void test_example__a_test_with_auxiliary_methods(void) |
| 312 | +{ |
| 313 | + check_string("foo"); |
| 314 | + check_string("bar"); |
| 315 | +} |
| 316 | +~~~~ |
| 317 | +
|
| 318 | +About Clar |
| 319 | +========== |
| 320 | +
|
| 321 | +Clar has been written from scratch by [Vicent Martí](https://github.com/vmg), |
| 322 | +to replace the old testing framework in [libgit2][libgit2]. |
| 323 | +
|
| 324 | +Do you know what languages are *in* on the SF startup scene? Node.js *and* |
| 325 | +Latin. Follow [@vmg](https://www.twitter.com/vmg) on Twitter to |
| 326 | +receive more lessons on word etymology. You can be hip too. |
| 327 | +
|
| 328 | +
|
| 329 | +[libgit2]: https://github.com/libgit2/libgit2 |
0 commit comments