Use Simple Checklists for Acceptance Criteria

Throughout this article, I’m going to use “GWT” as shorthand for Given/When/Then. And, in places, I will use “AC” as shorthand for Acceptance Criteria.

GWT is a popular canonical form for specifying requirements/tests. Some people like to use it for specifying the acceptance criteria of user stories. Dan North has a great article What's In A Story. In it, he presents an example that uses GWT to specify test scenarios, also using those scenarios as *the* acceptance criteria. There's a lot of great information in that article, except the bits about using the detailed test scenarios as the acceptance criteria. That is because they miss an important level of requirements abstraction for some stakeholders.

Let me try to be clearer... I *don’t* have a problem with GWT in and of itself. I absolutely love to use it for test scenarios. I am a fan of Behavior Driven Development (BDD) and Specification By Example (SBE), wherein you will find GWT extensively used. I also believe that test scenarios should be put into stories. I just happen to think that you should not jump from the high-level requirement detail of a story’s description into the low-level requirement details of a story’s test scenarios, because it leaves out an important level of requirements abstraction that is useful to a significant set of people in your audience, such as business stakeholders.

I believe the primary targeted audience for acceptance criteria are those people who do not need (or want) the excruciating detail of test scenarios, nor do they want some codified version of requirements. However, they do need more detail than they otherwise get out of a story’s title or description. These people are not going to read your GWT statements. They want something short and to the point, easy to understand. Here's an article that argues both sides. I think the argument against GWT, in favor of human-readable checklists, wins.

“So”, you might say, “are you arguing about the level of abstraction for acceptance criteria, or about using GWT for them, because those seem to be two different things?”. And you would be right. I am arguing about both. So, let me get to the next point: it is not a good idea to use GWT for acceptance criteria in the first place. At all

GWT is great at removing ambiguity, and it provides a common language. The canonical form is great. While these are all beneficial, they are unfortunately at the expense of readability and brevity. It is very important for the acceptance criteria to be easily interpreted by all stakeholders. And I believe the best way to do that is as a brief checklist. At this level, the acceptance criteria will be somewhat ambiguous. But that’s ok, because you absolutely should put test scenarios into a story as a means to back up the acceptance criteria with more detail, for anyone who wants or needs that level of clarity; you should use GWT to specify the test scenarios.

I have another reason not to use GWT for acceptance criteria: The ugly truth, in my experience with various story writers, is that most people are just plain bad at writing acceptance criteria. Here are bad practices that I see all the time:

  • In a voice other than the user's ("the code shall...", "the UI should...").
  • In terms of the implementation (referring to buttons, links, pages, screens, etc).
  • In terms of the tasks required to get the story done ("developer has fixed the problem").
  • In terms of assets that need to be created (documentation, tests).
  • Including things that are in the definition of done (unit testing coverage, etc).

From Characteristics of Good Agile Acceptance Criteria: “Acceptance Criteria should state intent, but not a solution (e.g., “A manager can approve or disapprove an audit form” rather than “A manager can click an ‘Approve/Disapprove’ radio button to approve an audit form”). The criteria should be independent of the implementation: ideally the phrasing should be the same regardless of target platform.

So the conclusion I draw from all of this experience is that bad AC with GWT is worse than bad AC in the first place. And the reason for this is that GWT makes AC harder to comprehend, because it’s wordy and a bit like reading code. There is more value in writing AC in a form that is readable in a non-canonical form, as a checklist, at a level whose detail is between the story description (As A...) and test scenarios.

And before you scold me about fixing this problem by simply teaching people to write better acceptance criteria... I get that, but I’m putting my realist hat on for this one.

To frame this another way, test scenarios are just the next (more detailed) level of requirement detail after the acceptance criteria. You get increasing requirements detail for each story, in the following order:

  1. Title
  2. Description
  3. Acceptance Criteria
  4. Test Scenarios

Let me give an example, stealing from Dan North's article:

A story title is a brief statement (E.g. "Customer can withdraw funds from ATM"). This can be thought of as a high-level requirement.

Next, we have a story description. It is a bit more detailed than the title, but also can be thought of as a requirement (E.g. "As an account holder, I want to withdraw cash from an ATM, so that I can get money when the bank is closed.")

And next, we have the acceptance criteria, also a form of requirement. E.g.

  • I must use my bank card to get money out of the ATM.
  • I can withdraw money as long as I ask for no more than is in my account.
  • I can withdraw no more than $400 at a time.
  • If I have more than one account, I can choose which account to take money from.
  • If there is a problem with my request, I am told about the problem and told how I can get help.
  • My card will be returned to me when I get my money.
  • My card will be retained by the ATM if it has been disabled.

Writing any of the acceptance criteria in GWT seems unnecessary to me, and even harmful. In my example, there are seven AC written on seven lines. If I put them into GWT format, they'll span about 25 lines. It just doesn't seem to make sense to me.

The last, most-detailed, level of requirement detail is the test scenarios. These are the most unambiguous (hopefully, if done well) set of requirements. They can be wordy; this is where GWT comes into play for me. Again, stealing from Dan North's example:

Scenario 1: Account has sufficient funds

Given the account balance is $100
And the card is valid
And the machine contains enough money 
When the Account Holder requests $20
Then the ATM should dispense $20
And the account balance should be $80 
And the card should be returned

Scenario 2: Account has insufficient funds 

Given the account balance is $10
And the card is valid
And the machine contains enough money
When the Account Holder requests $20
Then the ATM should not dispense any money
And the ATM should say there are insufficient funds
And the account balance should be $10
And the card should be returned

Scenario 3: Card has been disabled

Given the card is disabled
When the Account Holder requests $20
Then the ATM should retain the card
And the ATM should say the card has been retained

Scenario 4: The ATM has insufficient funds
The potential scenarios do not stop here; they can present themselves in a variety of sequences.

Side note: Tests are Requirement by Example. If you're getting hung up on the fact that I am saying that tests are requirements, I would recommend you read Bob Martin’s excellent paper, Tests and Requirements, Requirements and Tests: A Mobius Strip.