Testing

There are two types of testing implemented in pyro: unit tests and regression tests. Both of these are driven by the test.py script in the root directory.

Unit tests

pyro implements unit tests using pytest. These can be run via:

pytest -v --nbval

Regression tests

The main driver, pyro_sim.py has the ability to create benchmarks and compare output to stored benchmarks at the end of a simulation. Benchmark output is stored in each solver’s tests/ directory. When testing, we compare zone-by-zone for each variable to see if we agree exactly. If there is any disagreement, this means that we’ve made a change to the code that we need to understand (it may be a bug or may be a fix or optimization).

We can compare to the stored benchmarks simply by running:

./pyro/test.py

Note

When running on a new machine, it is possible that roundoff-level differences may mean that we do not pass the regression tests. In this case, one would need to create a new set of benchmarks for that machine and use those for future tests.