There was consternation in 345 testing towers back in early June...
…when our own “Mustang Sally“ Birch took us as for a “Chain of Fools” and published this article purely focused on Non-Functional Testing. I suppose we should admire her “commitment“, but it did make me feel like “Mr Pitiful“ so thought I would “Slip Away“ and write my own article.
OK, I’ll admit the article title is slightly misleading. Functional Testing isn’t really dead and this isn’t a eulogy. Instead, it’s an ode to how far reaching it could and should be, and a reminder of where the grey areas lie.
In the testing world, Functional Testing is king. Without it, Non-Functional Testing would be just “Non-“. In Sal’s article, even our own Non-Functional Test Architect tells us that Functional Testing is the foundation layer of her testing cake! (Note: I don’t really believe this – but don’t tell Sal, OK?!)
What is Functional Testing anyway?
Sal’s article suggests that Functional Testing is all about validating what the system does in terms of satisfying user requirements. Yes, I know, I said it, even in the brave new (Agile) world that we are often expected to live in these days, we are still ultimately validating user requirements (or user stories if that makes you feel more comfortable).
No IT system would exist without those pesky users requiring it to do something useful. So, as Sal says, it’s all about the system doing what the user wants, right? Well, yes … partially.
You see, Functional Testing is not just about validating that the system does what the user wants, it’s also about validating what happens when the system doesn’t do what the user wants.
To be clear, I’m not talking about the system being akin to a stroppy teenager who, when asked to do the washing up, refuses and retreats to their gaming console; more where they actually try to do the washing up but accidentally break your favourite mug on the floor while doing so.
Obviously, you, the user, didn’t want that outcome but that is what the system did. In fact, you probably didn’t even think of this potential outcome when you specified your requirement.
This is often a problem with IT projects: users are usually great at detailing what they want to happen (commonly known as the happy path) but frequently don’t think about the situation when that doesn’t or can’t happen (the unhappy path).
The Functional Tester's dual role
The Functional Tester must take into account both the happy and unhappy paths. For the unhappy path scenarios, the Functional Tester must identify and clarify the expected behaviour in each case. It’s the equivalent of having the foresight to ask “What do you want to happen if butter fingers over there drops your 345 mug on the floor?”
For the benefit of those who treat washing up as hypothetical, let’s consider a more real-world example. A large retailer may have several integrated systems, among them a Point of Sale (PoS) system to register sales of items and a local stock control system to monitor stock levels within that store and to trigger reorders if stock items become low.
A typical user requirement here might be: “When items are sold at the PoS, the quantity of each item sold must be transferred to the local stock control system and the relevant stock levels decreased”.
Sounds simple, right? When it works, yes. But the task here is to consider more than that. What happens if we travel the unhappy path? That’s when we have to answer questions such as:
- What if the stock level of an item on the stock control system is less than the number of items sold?
- What if the item doesn’t exist on the stock control system?
- What if an error is received when details of the sold items are transferred to the stock control system?
- What if no response is received from the stock control system when details of the sold items are transferred?
The grey areas of Functional Testing
Did you notice that the last two points above might not be counted as being part of Functional Testing? After all, as they relate to technical issues caused by the underlying technology, they can’t be considered functional, right?
Well, yes, I agree … sort of. In the purist testing view, this isn’t Functional Testing. But neither is it Non-Functional Testing either. So, who should test this when we’re in such a grey area?
Looking back at Sal’s article, you could argue that these types of issue are directly affecting what the system is trying to do but also how it is doing it. In our experience of many projects, this grey area actually becomes more of a black hole between Functional and Non-Functional Testing. These exception-type scenarios are sucked in only to be spat out again when the system goes to Production, often with disastrous results.
At 345, we subscribe to the age-old sci-fi adage: “Black holes are to be avoided at all costs.”
We believe in this so passionately that we spend more time focusing on these exception scenarios than we do on the actual specified requirements.
Don’t get me wrong, those happy paths are important but, typically, we find that it’s the unhappy paths that require more time and focus.
To put this into context, we have been involved in one large integration project for a financial institution where we created in the region of 20,000 test cases over the course of 6 years and where 95% of these tests were covering unhappy path type scenarios.
The Functional Tester's mindset
It takes a certain mindset in a tester to be able to identify these unhappy path scenarios as these aren’t necessarily obvious.
My wife would probably describe this mindset as “contrary and argumentative”, but I prefer to describe it as “focused and enquiring”.
A good Functional Tester approaches every scenario with one question in the back of their mind: “But what if that doesn’t happen?”. This can sometimes come across as being deliberately obstructive (ask my wife), but we encourage this behaviour within our testers as we have first-hand experience of what can happen if scenarios make it to that black hole. The unhappiest of unhappy paths is the one that is first discovered in Production at 2am in the morning! Been there, done that and learnt the hard way!
Our testers are the goalkeepers of our projects. Our job is to make sure that nothing slips through into that big black hole behind us. And, like all good goalkeepers, we take it personally if something evades our grasp, so, when we work with you, please don’t take offence if we raise our eyebrows when our questions draw responses such as “that would never happen” or “that’s not possible”. Trust us, from our experience, anything can and will happen, which is why we like to be prepared.
You may be wondering by now where our title about the end of Functional Testing comes into all this. In our view, there is no obvious end, no distinct line between Functional and Non-Functional. It’s all a bit of a blur.
This view helps us stop these scenarios slipping into that black hole. We do that by cheating (sort of) and using two goalkeepers – one Functional and one Non-Functional – who work closely together to achieve a quality outcome on every project.
