I’m an avid believer in testing - TDD helps drive design, and having a test suite available to verify behavior while maintaining an application is worth a million bucks. However, even the most complete unit test suite still doesn’t guarantee that the integration between different components is perfectly done, nor does it test the value a system delivers from a user perspective. Does a feature really do what a user expects it to do? Does your application fulfill it’s functional and non-functional requirements? Does it even run?
Acceptance testing tests features as if a user was interacting with them. This means testing the system through the user interface, a system driver layer, or a combination of the two. This could be useful in a couple of ways:
- As a smoke test for a deployed application.
- As part of your continuous integration build.
- As part of your deployment pipeline strategy.
I quite liked the following simple distinction between what BDD and TDD aims for:
To enable BDD, RSpec does a beautiful job in setting up context and Cucumber guides you down the given when then path. Both (as well as other testing frameworks) can be used to implement acceptance tests, but Cucumber only really makes sense if you have a business facing person looking at your specs. If not, writing your tests in English is a complete waste of time.
Unfortunately the .NET world does not quite have the same amount of tooling support for this kind of thing. This StackOverflow question gives a nice summary on the current state of BDD libraries for .NET. For the xSpec (context / specification) flavour, we have NSpec, Machine.Specifications, and NSpecify. Quite frankly, because of the constraints of a statically typed language, the syntax of all of these libraries sucks a little. We could try and use RSpec with IronRuby, but it will add an extra, unneeded paradigm in a project.
For the xBehave family we are in luck - SpecFlow, is a BSD licensed library that allows us to write specifications like Cucumber in the popular Gherkin language on .NET. SpecFlow integrates beautifully with Visual Studio, providing support for auto completion in feature files. Features compile down to NUnit tests, which can be run with your stock standard testing tool set.
I’ve written a small web application that allows you to search through an in-memory list of fruit, to test out the tool set:
This is what my feature looks like:
Behind the scenes, SpecFlow matches the Gherkin sentences to methods using regular expressions, which then drives the browser using Selenium backed by ChromeDriver for speed. For example, the following code drives navigation:
Note that if the path does not exist, yet the test is marked as pending and not failing - this allows us to write the specifications ahead of the actual code. We can of course also interact with the page:
In the spirit of Capybara, the code backing this will attempt to find the element by its id or the text of the label referencing the input element. This style of locating elements shows how we can write our tests from a user point of view. Another such example is finding text on a page:
For any kind of test to be repeatable, we need to have full control over the environment. This might entail setting up data, or even mocking out certain components in the application. Notice the setting up of the dataset for all of the tests:
In this example I’ve used the fantastic Deleporter library to reach into the running application and change its’ internals. This trick comes in useful when mocking dependencies that are difficult to control, like external web services.
My initial impression of SpecFlow has been excellent. It’s also nice to bring a little of the Ruby tooling into my day to day .NET work . I’m now trying it out on a much bigger project - stay tuned for lessons learned.
Some of this code is based on the official MVC example - you can find a complete example (and more) in the repository.