Dark theme

Geography Programming Courses

Doctests

Runtime bugs, that is, where the software runs but problems arise as it is running, are especially hard to find. Software houses spend a great deal of time and money getting people to try every combination of options to make sure all runtime bugs are found before software is released. As well as issues arising from code that is initially released, code can also break as new code is added. One of the ideas with good Object Oriented Programming, where functions and data are encapsulated in classes with a strong interface-based access, is that issues introduced in new code shouldn't break old code. Nonetheless, it would be good to have some way of confirming that code does what it should when we develop it and when we receive other people's code. The answer to this is testing. There are a broad range of ways we can test code, but one option in Python, which is quite nice, is doctests.

Doctests are short tests we can build into the documentation of a file that confirm the code is working as expected. They lack some of the sophistication of more complicated tests, but the nice thing is that they're built into the documentation and distributed with the code, while not adding anything extra to it. They can also be embedded into documentation (running doctests is, for example, one of the quickstart configuration options in Sphinx).

Here's some code with a test built in: test1.py. Download it and open it up. You'll see it contains the following in the docstrings for add():
>>> a = Calc()
>>> a.add(1,2)
3

>>> a = Calc()
>>> a.add(-1,-2)
-3
These are the tests that we'll run. You'll see they run the add method as if at the Python prompt, passing in some arguments, and then lists the expected result. When we run the tests the system we use will essentially run the code with the >>> prompts and check the results against the lines that contain the expected results in the doctests. Let's do that now. Open up a command prompt in the directory where you've saved the file and first run the code so you can see it works properly, generating a random result. Then enter:
python -m doctest -v docs.py
This runs the doctest module as an application, and passes in our file into it. The doctest module comes as part of the standard library.

After the random number associated with running the file, you should see the following output:
Trying:
    a = Calc()
Expecting nothing
ok
Trying:
    a.add(1,2)
Expecting:
    3
ok
Trying:
    a = Calc()
Expecting nothing
ok
Trying:
    a.add(-1,-2)
Expecting:
    -3
ok
2 items had no tests:
    docs
    docs.Calc
1 items passed all tests:
    4 tests in docs.Calc.add
4 tests in 3 items.
4 passed and 0 failed.
Test passed.
This is the tests running successfully. Now try altering the result of the first test in the file from 3 to 4 so the test is broken. Run the tests again. As well as the successful tests, you should see this:
Trying:
    a.add(1,2)
Expecting:
    4
**********************************************************************
File "C:\Google Drive\Gravity\gravity\practice\python\debugging\doctests\examples\docs.py", line 31, in docs.Calc.add
Failed example:
    a.add(1,2)
Expected:
    4
Got:
    3
**********************************************************************
1 items had failures:
    1 of 4 in docs.Calc.add
4 tests in 3 items.
3 passed and 1 failed.
***Test Failed*** 1 failures.
This is what a failed tests looks like. Obviously, usually this would be because the code failed, rather than the test being wrong, but the output is the same. Obviously we want to make sure our tests are correct! Change it back to 3 now, before we forget. We'll now go on to using tests to check code changes.