25 useful steps missing from SpecFlow
Over the last 5 years, we at endjin have tried to fully embrace Behaviour Driven Development (BDD). We've tried just about every framework that has been released into the .NET Ecosystem and we've even written our own when we found edge cases the others didn't support. Most of these frameworks have been based on traditional code centric unit testing; the major problem with these frameworks is that there is very thin line between intent & implementation.
One of our core mantras at endjin is to harvest as much IP as humanly possible, and every year we revisit our stack of IP, groom, refactor and consolidate the codebase in the light of any new additions, new constructs or discoveries which can improve or simplify the codebase.
One of the major pain-points has always been keeping all of the tests up to date because there are a myriad of frameworks, and because the behaviours described, in retrospect, are not as easy to understand as when they were written. There have also been occasions where we've had to port code across to a different platform (WP / Xamarin / Windows Store) where the BDD framework we used for that particular library was not available on the target platform. All of these friction points mounted up.
In 2010 Gojko Adzic published Specification by Example, which built on the concepts he had previously explored in Getting Fit with .Net Quick Introduction to Testing .Net Applications with FitNesse (2007) and Test Driven .NET Development with FitNesse: Second Edition (2009) and The Secret Ninja Cucumber Scrolls (2010) but raised the bar to focus on business value and collaboration between the development and the business teams.
This led me to revisit SpecFlow – a framework that has been around for a number of years, that enables Gherkin based specifications to be used on the .NET Framework. It was a pleasant surprise to discover that in the time since we had last used it to create HMRC compliant validation rules for a cloud payroll service, it had matured significantly and many of my original objections or friction points had been addressed.
It just so happened that this reassessment of SpecFlow coincided with creating a new leasing framework; we had been using an adaptation of Steve Marx's AutoRenewLease class, but wanted to support more scenarios and also make the implementation a little bit more light weight. Unfortunately the number of edge cases and possible conflicting scenarios for this type of framework are really very high, trying to describe the behaviour we wanted to create in terms of Machine.Specifications wasn't really working, so we swapped to SpecFlow.
We adopted a two step approach: we wrote as many different top level scenarios as we could to flesh out the behaviour, then slowly started to implement each scenario, once this was done we revisited the specifications to ensure they made sense and refactored accordingly. We ended up with around 28 scenarios spread across 3 features, many of which were implementation agnostic:
Scenario: Actor A attempts to execute a long running task whilst Actor B is currently running the task
Given the long running task takes 20 seconds to complete
And the lease name is "long-running-task"
And actor B is currently running the task
When Actor A executes the task
Then it should not throw any exceptions
And it should return successfully
And 2 actions should have completed successfully
This process worked out extremely well for us and we started using SpecFlow for all of our subsequent projects. After you've become familiar with SpecFlow, you start to realise that on a single project, as a team, if you were more consistent in your use of language you could achieve much more reuse of steps. Part of the refactoring process you have to adopt is to revisit all of the scenarios and see if you have steps that are variations of a sentence, if they are, you can easily adjust and reduce the number of custom steps you've written.
The next step is to realise that if you were more consistent in your language across projects you could factor out those steps into a reusable format and share them across projects; after all SpecFlow steps are just code and SpecFlow has some quite comprehensive extensibility points.
We examined Specs from a number of our projects and harvested the following 25 reusable steps into Endjin.SpecFlow:
[Then(@"the result count should be (.*)")]
[Then(@"the result should equal the datetime (.*)")]
[Then(@"the result should equal the datetimeoffset (.*)")]
[Then(@"the result should equal the integer (.*)")]
[Then(@"the result should equal the string ""(.*)""")]
[Then(@"the result should be false")]
[Then(@"the result should be greater than the datetime (.*)")]
[Then(@"the result should be greater than the datetimeoffset (.*)")]
[Then(@"the result should be greater than the integer (.*)")]
[Then(@"the result should be greater than or equal to the datetime (.*)")]
[Then(@"the result should be greater than or equal to the datetimeoffset (.*)")]
[Then(@"the result should be greater than or equal to the integer (.*)")]
[Then(@"the result should be less than the datetime (.*)")]
[Then(@"the result should be less than the datetimeoffset (.*)")]
[Then(@"the result should be less than the integer (.*)")]
[Then(@"the result should be less than or equal to the datetime (.*)")]
[Then(@"the result should be less than or equal to the datetimeoffset (.*)")]
[Then(@"the result should be less than or equal to the integer (.*)")]
[Then(@"the result should be null")]
[Then(@"the result should be of type (.*)")]
[Then(@"the result should be true")]
[Then(@"the result should contain")]
[Then(@"the result should equal the context value (.*)")]
[Then(@"the result should not be null")]
[Then(@"the result should not equal the string ""(.*)""")]
After this initial 25 we also found two reusable steps for dealing with exceptions:
[Then(@"an ""(.*)"" should be thrown")][Then(@"a ""(.*)"" should be thrown")]
If you would like to use these shared steps you can simple install the Endjin.SpecFlow package into your SpecFlow project:
Install-Package Endjin.SpecFlow
We have also created a SpecFlow extension for anyone who uses the Endjin.Composition Framework. This extension allows you to add the @container tag to any scenarios and it will initialise the Endjin.Composition container using all the installers it can find in the Specs project. You can use this library via NuGet:
Install-Package Endjin.SpecFlow.Composition
We also have a number of projects that require us to perform an integration test against a hosted endpoint or some static files. For these scenarios we created a reusable extension which embeds an Owin host into the specs project that enables you to run an in memory web application. You can signify that your scenario or feature uses this extension with the @web_app tag and can use the Library via NuGet:
Install-Package Endjin.SpecFlow.Owin.Hosting
All of these NuGet packages use custom XDT transformations to automatically insert the required entries into the App.Config of the specs projects. If you are thinking of writing your own set of reusable SpecFlow extensions, you may wish to reuse these files.
