(chibi test)

Simple but extensible testing framework with advanced reporting.

Testing

(test [name] expect expr)

The primary interface to testing. Evaluate expr and check that it is equal to expect, and report the result, using name or a printed summary of expr. If used inside a group this will contribute to the overall group reporting, but can be used standalone:
(test 4 (+ 2 2))
(+ 2 2) .............................................................. [ PASS]
=> PASS
(test "add two and two" 4 (+ 2 2))
add two and two ...................................................... [ PASS]
=> PASS
(test 3 (+ 2 2))
(+ 2 2) .............................................................. [ FAIL]
    expected 3 but got 4
=> FAIL
(test 4 (+ 2 "2"))
(+ 2 "2") ............................................................ [ERROR]
    ERROR: invalid type, expected Number: "2"
=> ERROR
The equality comparison is made with current-test-comparator, defaulting to test-equal?, which is the same as equal? but more permissive on floating point comparisons). Returns the status of the test (one of the symbols 'PASS, 'FAIL, 'SKIP, 'ERROR).

(test-equal equal [name] expect expr)

Equivalent to test, using equal for comparison instead of equal?.

(test-assert [name] expr)

Like test but evaluates expr and checks that it's true.

(test-not [name] expr)

Like test but evaluates expr and checks that it's false.

(test-values [name] expect expr)

Like test but expect and expr can both return multiple values.

(test-error [name] expr)

Like test but evaluates expr and checks that it raises an error.

(test-propagate-info name expect expr info)

Low-level macro to pass alist info to the underlying test-run.

(test-run expect expr info)

The procedural interface to testing. expect and expr should be thunks, and info is an alist of properties used in test reporting.

(test-equal? expect res)

Returns true if either (equal? expect res), or expect is inexact and res is within current-test-epsilon of expect.

Test Groups

(test-group name-expr body ...)

Tests can be collected in groups for Wraps body as a single test group, which can be filtered and summarized separately.
(test-group "pi"
  (test 3.14159 (acos -1))
  (test 3 (acos -1))
  (test 3.14159 (acos "-1")))
pi: .x!
1 out of 3 (33.3%) test passed in 0.00030422210693359375 seconds.
1 failure (33.3%).
1 error (33.3%).
FAIL: (acos -1)
    expected 3 but got 3.141592653589793
ERROR: (acos "-1")
    ERROR in "acos": invalid type, expected Number: "-1"

(test-begin [name])

Begin testing a new group until the closing (test-end).

(test-end [name])

Ends testing group introduced with (test-begin), and summarizes the results. The name is optional, but if present should match the corresponding test-begin name, or a warning is printed.

(test-exit)

Exits with a failure status if any tests have failed, and a successful status otherwise.

(test-syntax-error)

Accessors

(test-group-name group)

Returns the name of a test group info object.

(test-group-ref group field . o)

Returns the value of a field in a test var{group} info object. field should be a symbol, and predefined fields include parent, verbose, level, start-time, skip-group?, count, total-pass, total-fail, total-error.

(test-group-set! group field value)

Sets the value of a field in a test group info object.

(test-group-inc! group field [amount])

Increments the value of a field in a test group info object by amount, defaulting to 1.

(test-group-push! group field value)

Updates a field in a test group info object by consing value onto it.

(test-get-name! info)

Parameters

current-test-group

The current test group as started by test-group or test-begin.

current-test-verbosity

If true, show more verbose output per test. Inferred from the environment variable TEST_VERBOSE.

current-test-epsilon

The epsilon used for floating point comparisons.

current-test-comparator

The underlying comparator used in testing, defaults to test-equal?.

current-test-applier

The test applier - what we do with non-skipped tests. Takes the same signature as test-run, should be responsible for evaluating the thunks, determining the status of the test, and passing this information to current-test-reporter.

current-test-skipper

The test skipper - what we do with non-skipped tests. This should not evaluate the thunks and simply pass off to current-test-reporter.

current-test-reporter

Takes two arguments, the symbol status of the test and the info alist. Reports the result of the test and updates bookkeeping in the current test group for reporting.

current-test-group-reporter

Takes one argument, a test group, and prints a summary of the test results for that group.

test-failure-count

A running count of all test failures and errors across all groups (and threads). Used by test-exit.

current-test-group-filters

current-test-group-removers

Parameters controlling which test groups are skipped. Each parameter is a list of procedures of one argument, a test group info, which can be queried with test-group-name and test-group-ref. Analogous to SRFI 1, a filter selects a group for inclusion and a removers for exclusion. The defaults are set automatically from the environment variables TEST_GROUP_FILTER and TEST_GROUP_REMOVE, which should be comma-delimited lists of strings which are checked for a substring match in the test group name. A test group is skipped if it does not match any filter and:

current-test-filters

current-test-removers

Parameters controlling which tests are skipped. Each parameter is a list of procedures of one argument, a test info alist, which can be queried with test-get-name! or assq. Analogous to SRFI 1, a filter selects a test for inclusion and a removers for exclusion. The defaults are set automatically from the environment variables TEST_FILTER and TEST_REMOVE, which should be comma-delimited lists of strings which are checked for a substring match in the test name. A test is skipped if its group is skipped, or if it does not match a filter and:

current-column-width

Parameter controlling the current column width for test output, can be set from the environment variable TEST_COLUMN_WIDTH, otherwise defaults to 78. For portability of implementation (and resulting output), does not attempt to use termios to determine the actual available width.