Bridging the communications gap between clients and developers with Specification By Example.
Acceptance criteria
Acceptance criteria are the conditions that a software product must satisfy to fulfil its intended use - from that definition it's clear that acceptance criteria are critically important!
If the client and the developer have very different interpretations of the acceptance criteria there's a potential to produce site features that technically do pass the requirements defined in the acceptance criteria, but are not actually what the client wanted.
So how do we make sure we don't run into this problem? We produce acceptance criteria in the form of 'Specification by example'.
Specification by example
So before we get into defining a specification by use of examples, we should quickly look at what type of specification is defined with Acceptance Criteria and what Specification By Example defines.
Acceptance Criteria produces a requirement spec which is basically a description of requirements/conditions that must be satisfied.
Specification By Example produces a functional spec that is very similar to a requirement spec, but also includes examples that show the different ways in which the feature should behave when given specific values.
The value of Specification By Example is that it's a collaborative process that captures the requirements of a software project using realistic examples instead of abstract statements, in a format that's both easy to read and to convert into automated Behat test features.
A specification should focus on a single feature, with different aspects of the feature defined as scenarios of that feature. By focusing on a single feature your spec not only becomes easier to understand but also re-useable.
If you find yourself applying the same feature to multiple projects it's a lot easier if you have a clear spec for that site feature which has already been tried and tested.
Now we can look at what the foundations are for defining a Specification By Example.
Specification steps not test script steps
First things first, you need to understand that a specification and a test script are two completely different types of an automated test.
A test script is simply the journey a user must go through in order to achieve the goal of your software's feature.
If you were to try to automate this journey with a Behat test feature, it would be both simple and quick to automate, but would also be very fragile in terms of maintenance, and in general not actually useful as a spec or documentation, as it is just a test.
This is due to the fact it focuses on a user journey which is directly related to how the feature was implemented. Therefore it is just describing how the site feature should work.
We want to avoid this at all costs!
We never want to focus on how a site feature should work as it is too closely intertwined with how the feature was implemented. Your specification should not in anyway describe how a site feature was implemented but rather what it should do regardless of how its implemented.
Here are two examples of a Behat test feature:
Example 1
Feature: Free delivery
Given my shopping cart has a total of
Then the delivery fee should be
Examples:
|value|type|
|"£5"|"standard"|
|"£9.99"|"standard"|
|"£10"|"free"|
|"£15"|"free"|
Example 2
Feature: Free delivery
Given I search for "cheese"
And I add "big block of cheese" to my basket
And I add "small block of cheese" to my basket
When I click "checkout"
Then I should see my delivery fee is "free"
Both of these two features are testing the same site feature. So hopefully without any further information you can tell straight away that the first one is a specification and the other is a test script.
That's because the second test is a user journey, it lists the exact steps it takes to get free delivery. It uses specific values which again are directly related to how the feature was implemented.
So lets look at what details we actually get from just reading the first feature.
It doesn't mention any specific items that must be purchased just that the value must be higher than a certain number to be eligible for free delivery. From the examples listed we can work out than any basket with a total of £10 or more qualifies for free delivery. Anything below £10 will have a standard delivery fee.
This is specification by example. The Steps on their own explain what the feature should do rather than how it works. The examples provide real values that can be tested with an expected outcome.
This provides a clear spec for what a feature should do without specifying anything that implies a certain route for implementation, leaving the developer to make the decision on how to implement the feature in the most efficient and effective manor.
Which is exactly what we want. A mutual understanding between all stake holders but with no restriction to the solution chosen by the developer.
What are useful examples
There are many ways of extracting the right examples using various discussion techniques, I won't go into detail about those in this article, but I will list the qualities of good examples.
Easy to understand
This doesn't really need explaining but the key point is that these examples should serve as documentation and should not require any separate information.
Realistic
Examples have to be realistic, the main reason being that it will help you spot any inconsistencies before implementing a feature.
Precise
By being precise our examples help us avoid any uncertainty between stakeholders.
Complete
Make sure your examples include all edge cases, if we can define these all at the start then our technical solution can be built with them in mind. We never want to discover these edge cases midway through building as it can halt progress and potentially cause the solution to be completely re-designed.
Make sure to not over specify examples as it can potentially confuse the exact criteria of a specification or potentially add unnecessary instances of a test which will only add to the time it takes to run a whole test suite.
This may not seem like an issue but when you get to the point when your full test suite takes roughly 2 hours to complete, removing any examples that are not needed can save you a lot of time!
Show and keep quiet
Once you feel your examples have all of the above qualities and you've produced a clean specification by example, we recommend trying a process called Show and keep quiet.
The clue is in the heading - basically you should be able to hand your spec to a stakeholder, and without telling them anything, the spec should make enough sense by itself for them to understand what the specified site feature should do without needing to ask questions.
It may take a few versions of the spec to get it to the point that it is self explanatory but once you have achieved that, you will have clear documentation that you know doesn't require any attached information.
For more information on this topic
We recommend purchasing this book. Chapter 8 is especially useful.
Also why not take part in one of our Quality assurance training courses which cover this topic in more detail.