Testing & Quality Vision

Company wide test strategies can often be very procedural and rigid, which, in turn can lead to them not being followed as well as not utilising the full range of testing skills on a team. So, is there an alternative…? This article will look at why company test strategies may not work, what a testing and quality vision is and how they can utilise the skills of the team. Finally, it will go through metrics and how these can be used to measure the vision.

Teams

At some point in our lives, we have all worked as part of a team. That might have been at school sports day, a team building event or as part of a cross functional team delivering software solutions. Regardless of where that team originates from, I think it’s fair to say that everyone in a team has a different set of skills than the ones they need for their job.  For example, you could have a ‘manual’ tester who performs exploratory testing as part of their job but they may also have skills in requirements analysis.

If we take the example of that tester and let’s say the company wide strategy for testing was “everything shall be automated and there will be no manual testing at all”. Now that tester would not be able to use their skills to contribute to what their team produce and it may the in turn leave them disenfranchised and demotivated.

In examples like these the company test strategy doesn’t work because:

•   It may stop teams from utilizing the skills they have.
•  Members of the team may feel unfulfilled and not appreciated.
•   The team is not given the autonomy to decide the best way to test, they are having it forced upon them.

Now, teams can be complex with all the relationships within them, and every team has its own dynamic, which from the outside may look a little dysfunctional. However, successful teams know what works best for them and a bit of dysfunction is not always a bad thing. They usually know the best way to test a feature utilising the skills that they have to generate an outcome in which everyone is happy. Often imposing a companywide strategy upon them is not as good as letting the team decide how they want to test on a particular project or feature. Imposing anything on anyone may cause issues and telling a team how to test something is no different. This is where a testing and quality vision comes in.

Testing & Quality Vision

A testing and quality vision is a set of principles that define how testing and quality should look across the company. A team then takes that vision and comes up with a test strategy that implements that vision.

Example

So, what does a testing a quality vision look like? Well to be honest it can be whatever you want it to be, all it needs to do is reflect the testing and quality principles that you want your teams to adopt.

Here is an example of a testing vision with 3 principles:

1. Testing can be done quickly and efficiently.
2. Fast Feedback
3. Bug prevention over finding bugs

How your teams implement that vision is up to the teams themselves. Providing they adopt the various elements of the vision it really doesn’t matter. So how does the vision affect your teams? Let’s take this vision and 3 example teams and see what they could do to make sure that their test strategy aligns with the vision.

NOTE: I have just kept it to a couple of points, it would be a lot more in the real world.

Team Structure  - Developer with test automation skills and manual tester
Testing can be done quickly & efficiently	Tester writes BDD scenarios.	Tester involved in client requirement discussions to understand how feature can be tested prior to implementation
Fast Feedback	Developer & Tester pair test	CI Builds
Bug Prevention over finding bugs	Tester reviews pull requests.	Tester involved in client requirements discussions to understand requirements and highlight ambiguities

 

Team Structure - Developer with no test automation skills and automation tester
Testing can be done quickly & efficiently	Developer writes and implements BDD scenarios.	Automation tester writes/updates framework to help developer with their BDD scenarios
Fast Feedback	Developer performs exploratory testing on feature branch.	CI builds
Bug Prevention over finding bugs	Tester reviews pull requests.	Developer and tester involved in client requirements discussions to drive out examples that will feed into BDD tests.

 

Team Structure - Developer with test automation skills and no tester
Testing can be done quickly & efficiently	Tester writes and implements BDD scenarios.	Developer involved in client requirements discussions to drive out examples that will feed into BDD tests
Fast Feedback	CI builds	Pair programming
Bug Prevention over finding bugs	Pull requests have to be approved by another developer	Developer does exploratory testing on a feature branch

 

As you can see from the above, the test strategies would differ as they utilise the different skills set that each team has. There are obviously similarities (i.e. CI builds) as some principles are key, no matter what the team skill set is.

Now that is a very simple example but hopefully explains the concept.

Metrics

Now you have the vision and your teams have a strategy that adopts that vision, how can you check to make sure that your teams test strategies are aligned? One way is through metrics. Now metrics can be gamified, mis-interpreted and twisted so they cannot always be trusted. However, in this context I use metrics not as a stick to beat a team with but used to ask “Is there an issue here?”.  So, for example you could measure on average how long the CI build takes each sprint. In my example this will help with the “Testing can be done quickly and efficiently” vision principle. If the times shoot up, then that may indicate that there is a problem. However, it could be that after you have talked to the team, they inform you that the 3^rd party hosted agents that have been used were having issues so all builds took much longer than expected. This is something that is out of the teams hands. The metrics should be used to detect “smells” that could indicate an issue. It is import though that you talk to the team to understand exactly what is going on or to help them think about whether there is a problem and what that problem could be.

Here’s another example, say on the “Bug prevention over finding bugs” vision principle you use a metric to monitor this and you notice that the number of bugs raised is going up. You talk to the team, and you find out that they were doing some pair testing with the developer and tester but for some reason these have stopped. You can then try and understand why they have stopped and what needs to be done to get them going again. Its not a blame game it’s a way to help teams stop, take a step back and think and of there is a problem help them understand what they can do to solve it.

So, it’s really just a checkpoint to see if the team can identify a problem that they may have missed or a problem that doesn’t actually exist.

Benefits

So, what are the benefits of having a testing and quality vision and giving each individual team the autonomy to decide exactly how they test…

·       Utilises the skills of the team – The team can define their testing strategy by thinking about the skills of the team and how best to utilise those skills.

·       Empowers teams –  If a team can decide how they test, then it gives them the confidence to identify other areas where they could do the same.

·       Something you can measure – either through metrics or anecdotally from the teams themselves

·       Culturally drives improvement – By highlighting potential issues through metrics or anecdotally the team can learn to improve what they do and become more efficient.