I’ve been spending some time writing some of my first SpecFlow tests lately, and it’s led me to sorting out some assumptions I’ve made about acceptance tests. I’m sure I’m not finished.
It seems to me that an acceptance test is meant to do two things: First, it defines a feature. Second, it can be used to verify that feature works as expected. Perhaps it seems weird to call these two separate concerns, but bear with me.
Your goal when defining a feature is to clearly and simply describe it. The test serves as an artifact of specification, an agreement among the team about how the software is meant to work. Thus you want it to be human-readable. Not just developer-readable. Everyone on the team should be able to read it and understand it.
A Given-When-Then scenario with this goal in mind might look like this:
Given I am an administrator
When I go to the settings page
Then I see the administrative settings tab
Your goal when verifying the feature is ensuring that if a series of steps is taken, an expected outcome occurs. Ideally, I want to be able to automate my acceptance tests, and so when I’m writing them, I’ll start thinking about the specific steps that need to be taken to verify that a behavior of the system is working as expected.
A GWT scenario with this goal in mind might look like this:
Given I go to /Login
And I enter the following data into the Login form
| field | value |
| username | admin |
| password | foo |
And I click "Log In"
When I click the "Settings" link in the navigation
Then I am redirected to /Settings
And the administrative settings tab is displayed
So now we’ve written two different tests for the same behavior. One with humans in mind, one with automation in mind.
When I sat down to start writing these tests, I thought the latter was clearly the way to go. It seemed like, yeah, the balance was tipped slightly toward automation, but the human-readability was still there if you put in just a little bit of effort. I was also pretty excited at the prospect of doing GUI automation with WatiN, and I wanted it to be super easy to reuse a lot of our step definitions to automate our tests.
I was not prepared, however, for the level of detail involved in going step by step through some more complex workflows. I was also not prepared for the amount of redundant “code” I ended up writing. And perhaps most importantly: As I’ve written more and more of these style tests, I’ve been getting more and more terrified by how brittle they seem.
So I took a break to read up on it. And write about it a little. And now I’m going to let the pendulum swing back the other way. We’ll see what happens. I suspect some pain will come when I start trying to automate, and maybe I’ll get closer to the “right” level of detail for definition and verification.
