Is it better to write unit tests with a for loop, or to have each assertion separate?

Unit Testing involves breaking down a software code into small independent parts and examining those parts to ensure they work as expected. Unit tests should adhere to fundamental principles of clarity, readability, and maintainability. The question then remains: which is the best way to structure unit tests?

This question stokes a heated debate about how to structure unit tests, with most developers insisting that only one assert statement should be used in each unit test. In contrast, others argue that you can have more than one assert statement in a unit test. In general, both approaches, using a for loop or writing each assertion separately, have their places, and the best choice often depends on the context and the specific requirements of the test.

Using a For Loop

Some scenarios using for loops is acceptable as long as the tests themselves do not have complex logic. If a test has complex logic, who will write a test for the test? The scenarios where multiple assertions are acceptable are:

• When you have multiple sets of inputs that you want to test against the same logic or functionality, using a for loop reduces redundancy and keeps the code DRY (Don’t Repeat Yourself).
• When you have a large test data set, writing separate assertions for each can be tedious and error-prone. A for loop can help iterate the dataset and apply the assertions easily.
• When the test cases are not static and need to be generated dynamically, a for loop can be helpful to iterate over the generated test cases and apply the assertions. This, however, is a grey area since test cases should not be dynamic so that we do not introduce regression bugs into the test suite.

Consider the following Python test code for a simple function that takes two arguments, a and b, and returns their product multiply(a, b):

import unittest

def multiply(a, b):
    return a * b

class TestMultiply(unittest.TestCase):

    def test_multiply(self):
        test_cases = [
            (2, 3, 6),  # 2*3 = 6
            (4, 5, 20), # 4*5 = 20
            (0, 1, 0),  # 0*1 = 0
            (-2, 3, -6) # -2*3 = -6
        ]
       
        for a, b, expected in test_cases:
            with self.subTest(a=a, b=b, expected=expected):
                self.assertEqual(multiply(a, b), expected)

if __name__ == '__main__':
    unittest.main()

While using unit tests framework, the test can be executed in a for loop using the with self.subTest(a=a, b=b, expected=expected) subtext context used to create a sub-test for each test case. This way, if a test case fails, you will know which one it is, and the rest of the test cases will still be executed.

The for loop is acceptable in this scenario because we are testing the same logic, the multiply function, with different sets of inputs and expected outputs. The test cases are simple and concise, and using a for loop avoids redundancy and makes the test method more maintainable, especially if more test cases need to be added in the future.

Separate Assert Statements

This is by far the most acceptable method of writing unit tests. Unit tests are meant to follow a simple arranging, acting, and asserting (AAA) pattern. Each of these A’s should be visually separate, and this is what for loops tend to ignore. Consider the same code above multiply(a, b) used in the for loop section. We can rewrite that by separating them as shown.

import pytest

def multiply(a, b):
    return a * b

def test_multiply_positive_numbers():
    assert multiply(2, 3) == 6  # 2*3 = 6

def test_multiply_mixed_numbers():
    assert multiply(4, 5) == 20  # 4*5 = 20

def test_multiply_by_zero():
    assert multiply(0, 1) == 0  # 0*1 = 0

def test_multiply_negative_and_positive():
    assert multiply(-2, 3) == -6  # -2*3 = -6

if __name__ == '__main__':
    pytest.main([__file__])

Each test function represents a unique test case with a single assert statement with the name of each test function being descriptive, indicating what scenario it is testing. Each test function contains a single assert statement that checks whether the output of the multiply function is equal to the expected value for the given inputs.

In this case, single assert statements are acceptable since each test function is clear and explicit, making it easy to understand what is being tested. If a test fails, it is immediately apparent which scenario caused the failure, aiding debugging. Furthermore, the descriptive test function names serve as documentation, indicating the intended behavior of the function under different scenarios.

Parameterized Testing

Due to the constant debates about the best way to perform unit testing, a middle ground is achievable for testing the same logic with different input data. This is called parameterized testing and is more concise and readable. Consider the same multiply(a,b).

import pytest

def multiply(a, b):
    return a * b

# Parameterizing the test function with different sets of inputs and expected outputs
@pytest.mark.parametrize("a,b,expected", [
    (2, 3, 6),  # 2*3 = 6
    (4, 5, 20), # 4*5 = 20
    (0, 1, 0),  # 0*1 = 0
    (-2, 3, -6) # -2*3 = -6
])
def test_multiply(a, b, expected):
    assert multiply(a, b) == expected

if __name__ == '__main__':
    pytest.main([__file__])

Multiple sets of inputs and their expected outputs for the test function are defined using the parameterize decorator. This way of writing unit tests is more concise, readable, and maintainable and has the added advantage of clearly indicating which test has failed, which is usually the major issue with using for loops.

Conclusion

In conclusion, choosing the right approach depends on the context of the test, the specific requirements, and personal or team preference. A combination of these approaches is often used within the same project or test suite to address different testing needs effectively. Balancing conciseness, readability, maintainability, and clear failure output is key to crafting effective and efficient unit tests. By leveraging the features and capabilities of modern testing frameworks, developers can ensure the reliability and correctness of their software in various scenarios and configurations.