Handling a QA-less Development #1

It is not a secret that Vessel is being made with knowledge, passion, and a crippling absence of funding and fundamental team members. Today’s post will be about how we handle code quality in a development that doesn’t have any quality assurance expert. 

Crashing early

There are majorly two types of development philosophy, the first one states that the program (in this case, Vessel) should always be stable and reliable and it spends most of its efforts making sure the software always works. The second one consists of being overly dramatic and considering that every error must be fatal and crash the game. Believe it or not both schools can be useful in certain situations and development stages, and jokes aside, Vessel is using the second approach for now. 

The first school is really interesting for products that are already deployed, will have dedicated maintenance efforts, and have a dedicated QA team. It is really useful nowadays because it is far better for client facing software, no-one wants Netflix, Word, or Windows crashing. In order to achieve that, coders make sure to treat everything as a potential menace, make sure they have exceptions for every single case they can imagine, and a plan to recover from every situation. QA is essential in helping to define those cases.  But, we don’t have QA, we barely have a couple of coders so using that approach is not practical at all. 

The second approach is doing the absolute opposite. We crash Vessel as soon as we notice something is off. That has two main advantages:

1. It is really obvious that we ended up in a bad situation. As soon as there is a problem you see it.

2. Our bugs are, by far, easier to solve since they don’t progress much before being noticeable. 

Let’s see an example: 

We add a new weapon to Vessel and we forget to link a config file that specifies the weapon cooldown. If we use the first approach, we should have a default value that doesn’t compromise the game stability so it is a number that almost makes sense to keep the game being playable. Let’s say that the difference between both numbers is really small and barely noticeable. Let’s consider the default value is a cooldown of 0.12 seconds and the config file was supposed to set it up to 0.1 seconds. No one catches the error at the first moment. Later in the development a boss enemy that can only be killed if you deal an X amount of damage in a Y amount of time, is introduced. No one can beat this boss, suddenly the game difficulty spiked, worse than that, without noticing the team has created a deadlock because with the default values you can’t achieve the needed damage in the required time. 

If you are lucky and everything happens in a close timespan, you might solve the issue quickly. If you are not, this enemy can arrive months later than the weapon implementation and absolutely no one in the team will think that the issue is a misconfiguration in the weapon.

With the second school approach, you check if the config file is set at the weapon initialization, if it’s not, the game crashes. The issue is obvious. The situation doesn’t silently scale.

Crashing meaningfully

By the last example you might think that I am personally more in favor of the second school, and, in fact, I’m quite neutral so let’s see which issues have this second way:

1. If not handled early or managed in any way you can block the whole team by any mistake no matter how stupid or small it is. 

   1. This can be mitigated with feature flags (I know you remember them from a previous post).

2. Different execution environments might raise false issues. I.e. Exiting the game in the editor might delete some data in a way that doesn’t happen in a packaged game causing crashes that won’t exist in the game.

3. Crashing makes it obvious that there is an error but not always makes the error itself obvious.

For the second and the third issues the best way to solve them is writing meaningful crashes. Doing that is as simple as providing clear information on what happened, and even more clear instructions on how to solve the issue (if it’s solvable).

Revisiting the prior example, if the message is some cryptic code-like assertion, only coders will be able to understand it. But if it states something like “Weapon_A failed to initialize due to a missing configuration file in the field CooldownData. Please make sure the setup is correct!” anyone with access to the editor can fix the issue. If the message is clear enough it won’t matter where or when it crashes the fix will always be easier to solve.

That’s the main reason the Vessel team is using this approach. It ensures that we won’t have any silent issue that only an expert QA can detect. We make the errors obvious, and make our crashes meaningful to ensure we are always informed about what is happening. Does it mean we don’t need QA? Absolutely not. But we can push QA entry for a later stage when, hopefully, we will have more money.

As you might have already deduced, this is the first one of a series of posts where I will be explaining how we are dealing with QA absence. Stay tuned for the next entries! Please share the post, drop a like, and if you like this type of content consider donating to help keep this page alive!