C#, Span and async
C# 7.2 introduced support for ref struct types. These are types that are required to live on the stack. The most notable ref struct types are Span<T> and ReadOnlySpan<T> which have made it possible to dramatically improve the efficiency of certain kinds of work by reducing or even eliminating object allocations.
Endjin recently open sourced our AIS parser at https://github.com/ais-dotnet/ and it relies on these techniques to make it possible to parse millions of messages a second on a single thread, all without any per-message memory allocation. (AIS is the international standard by which ships report information including location, speed, and heading. We wrote this parser as part of a recent project with OceanMind. This video provides information about the kind of work OceanMind does.)
There's a tradeoff with these techniques: ref struct types can only live on the stack. They rely on the .NET runtime's support for managed interior pointers. These are special kinds of pointers that are allowed to point into the middle of an object—at a particular field in an object or a particular element in an array—and if any of these pointers are live, they will keep their target object alive just like a normal object reference would. (A normal object reference always points to the start of the object.) But they are more complex for the GC to manage, so it only supports managed pointers that live on the stack.
This stack-only requirement obviously means you can't write a class with instance fields of any ref struct type—a class's instance fields live inside the object's memory on the managed heap. More subtly you can't use these types for fields in an ordinary struct either—although value types often live on the stack, they don't have to: if you use a value type as a field in a class, or an element type of an array, the values will end up in a heap block. (And of course, if you box a value type, it ends up being copied into its very own heap block.)
The only types that can have fields with types such as ReadOnlySpan<T> are other ref struct types.
Iterators, async, and ref structs
A problem you're highly likely to run into if you use ref struct types much is that the stack-only limitations come into play whenever C# turns your local variables into fields. It does this with any async method for example. (And also for iterator methods, and for the same reason: both kinds need local variables to be able to survive after the method returns, because these methods can typically return several times during the course of their execution, later picking up again from where they left off.)
The upshot is that you can't declare a ref struct-typed local variable in an async method or an iterator. That means that this won't work:
This tries to use the NmeaAisPositionReportClassAParser from our Ais.Net library. That type uses ReadOnlySpan<T> internally as a field, meaning that it too has to be a ref struct. So you can't declare a variable of this type in an async method.
However, it's relatively straightforward to work around this, using the technique shown here:
I've moved the code that uses the ref struct type into a local method. That local method is not async meaning that you can't use await inside it, but you can have ref struct variables.
Although C# has to generate a hidden type to give the local method access to variable declared by the outer method (e.g., line in this example), this doesn't cause any extra allocations. The hidden type will be a struct in this case, and it will be passed to the local method as a hidden ref argument, avoiding either unnecessary copying or object allocation. (Be aware that if you create a delegate that refers to a local method, C# is no longer able to perform that optimization, because it needs to ensure that the local variables are able to outlive the outer method, just in case you try to use the delegate after returning. But for this scenario, that problem does not arise.)
This technique also works for iterator methods. And of course it works for .NET's own ref struct types such as Span<T>.
