TL;DR - This series of posts shows how you can integration test Azure Functions projects using the open-source Corvus.SpecFlow.Extensions library and walks through the different ways you can use it in your SpecFlow projects to start and stop function app instances for your scenarios and features.
If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. Even now, several years after their introduction, the testing story for functions is not hugely well defined. If you're building your functions well, then there won't be a lot of code in them - they will be thin facades calling into code that does the bulk of the work, in which case you will likely have used a standard unit testing approach on that code. Nevertheless, that likely leaves some functionality untested - for example, ensuring your models are correctly bound to input, and ensuring that correct status codes, headers and so on are returned from your requests.
As such, it becomes necessary to step up a level and look at how to test the functions as a whole. There are two options for this:
- You can test in-process, using the approach defined in Microsoft's docs (note that this doesn't seem to have been updated to take account of functions that use instance methods, but that's unlikely to affect the approach). This is good, but doesn't ensure that your function is configured correctly, and if you're using automatic model binding, it doesn't test that this is working as you expect.
- Alternatively, you can test out of process, either against a deployed instance of the function or against one that's running locally. Testing against a deployed instance is a great idea, but this is normally reserved for another level of testing, meant to ensure that things are working as expected in a deployed environment. It doesn't address the needs of the developer as the feedback loop from making a change to deploying a function to Azure is likely just too long. This leaves us with the challenge of testing against a function running locally.
So, how do we go about this?
Before I continue I should note that while I'm specifically addressing how to do this with SpecFlow, a very similar approach can be taken with other frameworks. However the code I share won't immediately work elsewhere because it's implementation depends on SpecFlow constructs for storing and retrieving state.
As always, it's worth starting with what we want to achieve:
- We want a way of automatically starting a function, and then shutting it down once the test is completed.
- We want this to work in as close a way as possible to a deployed function
- Ideally, we want to be able to capture the output from the function while it's running.
- It's useful to be able to easily affect the configuration of the function under test.
- We want an approach that can work as part of a CI pipeline.
So, let's have a look at how we achieve these goals.
Running the function locally
When you hit F5 to run a function in Visual Studio, it uses a copy of the Azure Functions Core Tools that's managed by Visual Studio. Normally they get automatically installed into
C:\Users\username\AppData\Local\AzureFunctionsTools\Releases and Visual Studio selects the correct version to used based on your project's runtime.
However, this is an internal detail of how Visual Studio implements the Functions SDK, so it's not really something we can rely on. Fortunately you can install and use Azure Functions Core Tools directly.
NOTE: From here on in, I'm going to assume you're developing using Functions v3 and dotnetcore3.1. If you're using a previous version you'll need to adjust the commands and code I'm showing, but it should be pretty obvious how to do so. If you've got any questions just leave a comment below, or raise an issue on the Corvus.SpecFlow.Extensions project.
To get the tools installed, you'll need npm. Run the command
npm i -g azure-functions-core-tools@3 --unsafe-perm true
This will install the tools locally - you can verify they are there using the new
func command from the command prompt. If you do this, you'll see all the things you can do with it - scaffolding new functions apps and functions, and running functions locally. The latter is what we're concerned with - you'll see that you can start a new function using the command
func start, providing port number and other details as part of the command. This is what we're going to use when setting up our test.
The code to start, stop and manage functions as part of a SpecFlow test is part of the endjin-sponsored Corvus.SpecFlow.Extensions library. The classes that do the bulk of the work are:
FunctionsController.cs - this contains methods to start a new functions instance, and to tear down all functions it manages. It's intended to live for the lifetime of the test as it captures the output and error streams from the function and write them all to the Console when the functions are terminated. When running in SpecFlow, this results in that information being written to the test's output.
FunctionConfiguration.cs - this is part of the mechanism by which the test project can provide settings to the function under test.
FunctionsBindings.cs - this provides a couple of standard step bindings that can be used as part of a scenario to start a function.
This code is all open source, and contributions are accepted. It's available under the Apache 2.0 open source license meaning you're free to use and modify the code as you see fit. The license does impose some conditions around retaining copyright attributions and so on - you can read the full details here.
This code ticks the boxes for the first four of the five goals I set out above, providing mechanisms to keep functions running for the duration of test execution, as well as a way to supply additional configuration. The next few sections explain the different ways of using this.
I'll be doing this with reference to the demo project that's part of the Corvus.SpecFlow.Extensions codebase. Before continuing, I recommend downloading the project so you can examine the code. There's a functions project, Corvus.SpecFlow.Extensions.DemoFunction, that contains a slightly modified version of code that's generated when you create a new HTTP-triggered function in Visual Studio. It accepts GET and POST requests, looking for a parameter called
name in either the querystring or request body, and returning a configurable string containing that parameter.
It also contains a SpecFlow test project, Corvus.SpecFlow.Extensions.Demo, with a folder called AzureFunctionsTesting containing 5 feature files which relate to the following next few posts in this series.