Integration Testing Azure Functions with SpecFlow and C#, Part 3 - Using hooks to start Functions
TL;DR - This series of posts shows how you can integration test Azure Functions projects using the open-source Corvus.SpecFlow.Extensions library and walks through the different ways you can use it in your SpecFlow projects to start and stop function app instances for your scenarios and features.
In the first two posts in this series, we introduced the Corvus.SpecFlow.Extensions project and showed how you can use the SpecFlow step bindings it provides to start functions apps as part of your scenarios. This approach has the drawback of making your scenarios harder to read for non-technical users, so in this post, we're going to take a look at using scenario and feature hooks to address that problem.
Using per-scenario hooks to start and stop functions - as shown in ScenariosUsingPerScenarioHook.feature
SpecFlow hooks allow us to add code that's executed at specific points during a test run. With this method, we make use of the BeforeScenario and AfterScenario hooks, and use the Corvus.SpecFlow.Extensions.FunctionsController class directly to start and stop our functions. This can be seen in the DemoFunctionPerScenario class:
The parameters that the StartFunctionsInstance method takes are the same as those shown in the step binding above, allowing you to specify project, port and runtime. You'll see that the create FunctionsController instance is stored in the ScenarioContext; this allows us to pull it out in the AfterScenario method (which you should add yourself, as shown in the test code) to tear down the functions.
Advantages to this method
Using this method conceals the technical detail of what the setup step involves, reducing it to a single tag for the function. This makes your scenarios much more readable. If you're writing lots of tests for a specific functions app, it also reduces the duplication needed when every scenario has to contain the setup step.
In addition, test output (and the associated functions output) can be viewed in exactly the same way as above.
Disadvantages to this method
The main disadvantage to this approach is one that's associated with pretty much all integration testing: speed. Whilst setting up and tearing down all dependencies for each test is the gold standard, spinning up functions takes time and this can mean your test suite takes a long time to execute. In some scenarios this may be unavoidable. However, others may lend themself to using the third method to strike a balance between speeding up execution and isolating tests.
Using per-feature hooks to start and stop functions - as shown in ScenariosUsingPerFeatureHook.feature
Visually, this approach looks extremely similar to the previous method. The scenario definitions are not affected at all and the only differences in the underlying code (other than using BeforeFeature and AfterFeature attributes) being that the FeatureContext is used in place of the ScenarioContext to store and retrieve the FunctionsController. The other difference is that the hook methods themselves need to be static to be used with per-feature hooks - this is a SpecFlow requirement.
Advantages to this method
If you can group related scenarios and be sure they won't conflict with one another, this is a relatively easy way of speeding up test execution.
Disadvantages to this method
As implied above, this approach does have the potential to cause unexpected results if your tests conflict with one another in any way. The other disadvantage is that because the function output is gathered and written to console when the function is terminated, it can no longer be seen in the test output. If you don't mind duplication, you can get round this by adding an additional AfterScenario hook to write the function output to the console after every scenario.
