Introduction to Pytest Framework

Testing projects with unittest and Pytest

In Python, there are two well-known stable libraries for testing: unittest , which is the official testing standard library, and pytest , which we’ll be looking at. Note that Pytest supports tests written via the unittest library but not vice versa.

In unittest , we make a class and inherit from unittest.TestCase and add the tests as methods to that class.
Consider a function called divide(x: int, y: int) which is responsible for dividing the y value from x and returning the result.

def divide(x: int, y: int) -> float:
  return x/y

Simple, isn't it? Now, let's write a test case for that function via unittest in Python.

import unittest
from package.module import divide
class MyTestCase(unittest.TestCase):
def test_divide(self):
self.assertEqual(divide(15,5), 3)

To run tests, run the following command in a command-line prompt. It will browse through the packages and find tests and run them instantly.

$ python -m unittest

Here, we have designed a custom TestCase called MyTestCase and added the only test we need, test_divide(self). We've put this class in tests/module_test.py.

As seen, there is an OOP structure behind all tests implemented with unittest library. On the contrary, Python pytest supports both functional and class-based ways of test implementation.

Note: Make sure that test files are in either ".py" patterns otherwise most testing tools will not be able to find them.

Some people prefer to use the _test.py pattern. It makes it easier to type the first few letters of the file and press TAB to find the test file instantly.

Note: To make tests/ discoverable by unittest or Pytest, you need to have an empty "init.py" file inside the tests/ directory. This way, unittest will deal with tests/ as a package and will seek testing files from that directory as well.

Remember to name the test methods/functions in test(...) pattern.

The example written earlier is a classic way of writing Python unit tests via unittest. Now, we’ll look at Pytest. In the next two sections, we'll implement the same MyTestCase in Pytest.

How to use Pytest

Pytest is famous for its smart discovery and parallel test-running features. It is an external Python package, meaning it needs to be installed first using the command:

$ pip install pytest

If we run the pytest command in the same directory that we ran python -m unittest command in, Pytest will start discovering the test files. Since the tests made with unittest module are part of Pytest's discovery, we assume Pytest discovered it first and runs it.

$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
rootdir: /private/tmp
collected 1 item                                                               
tests/module_test.py .                                                    [100%]
============================== 1 passed in 0.00s ===============================

Now, let's rewrite the test, but in Pytest.

# tests/module_test.py
from package.module import divide

def test_divide():
assert divide(20, 4) == 5

Pytest is assertion-friendly. Simply write the testing functions and add assertions into them and Pytest will count them as tests.

There’s a nifty feature of Pytest that enables us to enter multiple values as inputs as well as the expected results. Pytest will test the function with every single entered value and check the results compared to the expectations.

In the following code, we've extended our previous test_divide() test function. It now includes three related tests inside itself.

# tests/module_test.py
import pytest   # new
from package.module import divide

@pytest.mark.parametrize(   # new
"x, y, expected", [
(10, 2, 5),
(100, 10, 10),
(20, 5, 4)
]
)
def test_divide(x, y, expected):   # new
assert divide(x, y) == expected   # new

If we run the pytest command again, we’ll see that three tests have been passed.

$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
rootdir: /private/tmp/my-package
collected 3 items

tests/module_test.py ...                                                 [100%]
============================== 3 passed in 0.01s ===============================

We can also turn this function into a class that does the same thing (class-based).

class TestModule:
@pytest.mark.parametrize(
"x, y, expected", [
(10, 2, 5),
(100, 10, 10),
(20, 5, 4)]
def test_divide(self, x, y, expected):
assert divide(x, y) == expected

Always be aware of the following naming conventions throughout the testing process.

• The class name has to start with Test* name.
• The class method has to start with test* name.

Now, let's extend our division function (not the test) and handle the ZeroDivisionError exception.

def divide(x: int, y: int) -> float:
if y == 0:    # new
raise ZeroDivisionError('y value must not be 0')   # new
return x/y

To test that specific ZeroDivisionError occurrence, we add a new test in module_test.py.

# tests/module_test.py
...

def test_zero_handling_exception():
with pytest.raises(ZeroDivisionError):
divide(20, 0)

We can also add tests to ensure that some part of the program fails when inappropriate values are entered as inputs. This is called exfail (expected to fail). We can specify the exact reason for the failure as well via raises and reason.

Here are some examples of exfail test cases:

@pytest.mark.xfail(raises=ZeroDivisionError)
def test_divide_by_zero():
divide(100, 0)

The result is:

============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
rootdir: /Users/sadra/my-package
plugins: mock-3.10.0, xdist-3.3.1
collected 5 items

tests/module_test.py ....x                                                [100%]
========================= 4 passed, 1 xfailed in 0.02s =========================

Test structuring

In this section, we’ll explore the proper structures to observe to keep tests in projects. Look at the following package tree and focus on the directories that are related to testing.

my-project/
├── package
│   ├── __init__.py
│   └── utils.py
|   └── module.py     <-- divide() lives here
├── ...
└── tests
    ├── __init__.py
    ├── utils_test.py
    └── module_test.py

The best practice is to keep tests in another package within the project. This way, the tests will have nothing to do with the main structure of the project. We want tests to be isolated and not violate any other part of the project.

As for the components that the tests/* might need it in order to work properly, it’s recommended to keep the requirements of the tests inside another separate package called testing/ next to the tests/ directory.

my-project/
├── package
│   ├── __init__.py
│   └── utils.py
|   └── module.py
├── testing           <-- new
│   ├── __init__.py
|   └── ...
└── tests
    ├── __init__.py
    ├── utils_test.py
    └── module_test.py

Keep in mind that testing/ does not contain any testing script or anything executable by any testing tool. It only contains the modules, tools, classes, and functions we might need in tests/* to test our project's main functionalities.

Automation

A necessary practice when testing in Python is having automated pipelines to test every single piece we add or modify. That said, Python developers can use a number of tools for the same.

We can have complete control over our Pytest hook inside the pre-commit configuration, meaning we can test every single commit we make.

On most CI tools, it’s possible to test contributors' changes that are on the verge of being integrated into codebases. For instance, we can run Pytest tests, output a coverage template, and run them per each pull request that people make on the repositories.

Conclusion

The Pytest library provides a vast range of features and functionalities as well as support for recognizing test modules and files, compared to the official unittest library. Since Pytest is supported by many testing automation systems and code analyzers, it’s a solid choice for safely writing isolated unit tests in different environments. Try the test outlines in this Pytest tutorial and see how they work for your project.