Testing Methodology ¶

There are two common scopes for testing:

• 🛸 code coverage: we write tests covering every path (if/exceptions/...) that the code could take.

• 🔎 unit testing: we test a part of the code, usually a function

➡️ There are many tools to test the coverage of your code, which are slowly being added to the Continuous Integration page.

When writing tests it's important to test both correct cases ✅, and incorrect cases ❌. This is to ensure that when updating the code to add new features/fix bugs, the correct cases are still correct.

• ⬜ White-box testing: you know about the code
• ⬛ Black-box testing: you only know about I/O (params/result)
• 🌫️ Grey-box testing: a mix of both

RightBICEP ¶

This approach aims to identify effective test cases:

• Right: test that the output is correct
• Boundary: test boundaries/extremum (ex: MAX_INT...)
• Inverse: test the inverse input/output (ex: 3+2 is the same as 2+3)
• Cross-check: cross-check the output with another function
• Errors: test error cases
• Performance: test performance and resource usage

CORRECT ¶

This approach also aims to identify effective test cases:

• Conformance: test that the I/O conforms to what was expected
• Ordering: test that I/O are in the correct order
• Range: test boundaries/extremum
• Reference: test against a known result, test that the correct reference/object in memory is used
• Existence: test that the required data/result are defined (not null/...)
• Cardinality: test with 0, 1, and more arguments (if applicable). Test with items of different cardinalities (arrays/lists/...).
• Time: test timing and performance. Test sequencing (login→logout).

Assuming that

• preconditions: requirements before the execution
• postconditions: expected result after execution
• invariants: properties that must remain unchanged

Design by Contract ¶

Design by Contract (Conception par contrat) is an approach to software development. We will check the preconditions (ex: before executing the rest of the function), and the postconditions after calling the function (ex: before returning the result, or after calling the function). If a condition fails, then the function has a defect.

➡️ Ex: using assert to test parameters/the result

Contract programming ¶

Contract programming (Programmation par contrat) is a programming paradigm. We declare a contract associated with a function, such as "when given two integers, it will return an integer".

If we call the function with preconditions defined in the contract (e.g. two integers), and the result is invalid, then the function has a defect. The function can be called with invalid preconditions (as per the contract, such as a float and an integer), but there is no guarantee of the outcome.

➡️ Ex: using @Contract (Java), @contract from contracts (Python).

Defensive programming is implemented by adding checks to prevent errors or unexpected outcomes.

👉 Defensive programming is often used along contracts, to implement error handling and input validation.

➡️ Use IllegalArgumentException (invalid parameters) and IllegalStateException (invalid result, invariant...) in Java...

• check if objects/addresses are null/undefined/...
• check the range (RGB: [0,255] that may be stored in an integer)
• check the size, the type, ...
• check runtime conditions (is login called before logout?)

👉 Apply Zero trust principle: do not trust any input/output, check everything.

Behavior-driven development (BDD) is a software development methodology that breaks down every part of the code into 3 sentences, explaining concretely how the code should behave.

👉 These questions are answered for every use case of a function before adding the function to the code base.

Test-driven development (TDD) is a development practice in which we write the tests before the code. For instance, in Java/OO,

1. we would write a class with an empty method raising an exception
2. we will write tests for this method
3. then we will code the method

👉 You will code the method to pass one test, then another one... while ensuring that previous tests are still OK.

➡️ It's hard to apply this method when starting from scratch. You should first design your project (ex: UML), then generate code from the UML, and start from here (better: apply Model-driven development too).

You should always ask yourself

• What is my function supposed to do?
• Will adding my function, do what we want?
• What problems might be caused by adding my function?

• A/B testing
• SOLID
• JBehave BDD framework (Java)