Matlab + Octave unit tests quick-start

Matlab unit test framework is well-established and said to have a JUnit-like structure. Matlab’s unit test framework is completely incompatible with GNU Octave’s unit testing framework. A common ground between Matlab and GNU Octave can be found in the easy to use Matlab script-based tests, which can be run as an ordinary script with GNU Octave.

Script-based test

A key characteristic of Matlab script-based tests is they can’t use the Assertable functions. This also means you cannot mark a test as skipped or incomplete with script-based tests. This is not generally a big issue unless your project specifically mandates such usage.

Matlab or Octave can run a script-based test as a plain script. The downsides of running a script-based test as a plain script are:

  1. there is no decorated TestResult
  2. the first failure ends the whole test

It’s best to put the script-based test scripts in the same directory as the code they’re testing to avoid Matlab path issues.

Matlab runtests

When using Matlab on this Octave-compatible script-based test, a richer result comes from using Matlab-only runtests(). Matlab runtests() will search all subdirectories under the current working directory and run all scripts with case-insensitive “test” in the filename. For example, a script testing HDF5 function of a program might be named TestHDF5.m or HDF5test.m. This is similar to PyTest filename-filtering.

Denote each test in the script-based test file with a double percent sign like:

% test_example.m
% setup code goes up top, as each test's variables are isolated from each other
% when using Matlab runtests()

A = magic(2);

%% test_dummy
B = A*2;  % only visible to this test, when using Matlab runtests()
assert(isequal(B, A*2), 'dummy example')

%% test_four
C = A*4; % note that "B" is not visible under runtests()
assert(isequal(C, A*4), 'dummy four')

Run Matlab runtests() from CI command using a runner script. Otherwise, just doing matlab -batch runtests will NOT fail in CI, even with failing test.

Octave

Octave runtests('.') only runs tests in specially formatted comments that Matlab ignores. Thus, we generally recommend writing Matlab script-based tests, and manually running each file from Octave.