Handling a QA-less Development #3

Welcome back! The last posts have been really exciting and I’m afraid this one is not going to keep pace. This week there are no big revelations, just development insights. As you might have noticed this will be a technical post. This time I want to talk about testing. 

“How are you testing things if you don’t have QA?” That’s a good question. Well, essentially we write extra code to test our gameplay code! Hold on, don’t run away yet,  it might sound scary but the truth is the concept is quite easy. 

Think about it like having a solution sheet for some exercises, even if you don’t have the teacher you can check if your solution is right or wrong. The core idea is the same, we don’t have QA to test things but we can check a small bit of code and see if the output it gives us is the one we expect.

Defining what to test

There are a lot of different ways to approach code testing and a lot of great authors that can go into the details better than myself. But before going into more details on how we are planning to use those I think it’s good to explain a couple of core concepts.

The first one is defining what we want to test. Is it every line of code? Every class? Every function? These are usually referred to as testable units. Depending on the code purpose we might want to test smaller units or bigger ones. Again there are a lot of different schools on how to do this properly, but the key here is to build enough tests to achieve our purpose but not too many since they will have a cost. They are more code, therefore the team needs time to implement those and to maintain them.

For me it depends on the code responsibility. If I’m coding a medical device that might put people's lives on the edge, or I’m building a critical software that might cost millions if it fails, then I’ll probably make sure that each line of code is tested. On the other hand, if I’m building a website or a cosmetic piece of software I would not be burdened to test each line, but maybe each component of that software. 

Defining why are we testing

Once we have understood what a testable unit is, we need to think about why we are placing the tests. Do we want to validate how the implementation works  or do we want to check if the implementation is correct? They might sound similar but there is a huge difference. 

Validating the implementation is just checking that we receive the correct answer. It doesn’t matter if it has been found by luck or by knowledge. We don’t care if the code is running on thoughts and prayers as long as we receive what we expect. 

Checking how the implementation is actually done though doesn’t mean we are validating it. A text might sound right, fit the word counting, being well justified, and yet mean nothing. For example the “Lorem ipsum” text might pass an implementation test but still be a garbage if we try to validate its meaning. Same happens with code, we can check that the solution performs within a time frame, and respect few parameters but this won’t mean it’s giving us the correct outcome. 

“Why not get both then?” Excellent question! Again is a matter of costs, the more bounded the tests are to a certain implementation the most likely is that we need to change them if we ever change the code. 

How are we intending to test in Vessel?

Well, first of all we are using tests to protect code from our future selves. Since the team is already doing peer reviews to ensure that the implementation is correct and up to the standards, we just want to ensure that the testable units are validated.

Let’s analyse the specifics of the project. Vessel is a composition driven code so as a testable unit we are using our components. Since we are not evaluating how those are coded we can treat them as a blackbox. Each component will be able to receive a series of inputs and will produce a series of outputs. In example, a health component will have a Modify function that will receive a numerical value. So a test in this case will consist of modifying this value by a given amount and see that the result is what we expect. Once we have placed the test we try to send different values and see if the results are still consistently what we expect. We usually use a good value, an ill formed one, and the limits values. 

Those tests might seem stupid to be built once the component is already done and the feature validated, but our aim is to protect the code for the future. Let’s skip to next year for an example: The game is already done and is selling well. We are moving to the second development and suddenly we decide that we are going to have shields. The coder decides that the best way to do it is to reimplement the health component and take shields into account. We can run the tests for this component and make sure it is still producing good results. If so it means that the integration won’t break anything else on the codebase. If those tests fail it means that we are breaking systems. 

What do you think about this strategy? I will read you in the comments! 

PS: If you want to dig more into this check out Unit testing principles practices and patterns from Vladimir Khorikov. Absolutely delightful.