C# 8.0 nullable references and serialization

When you enable C#'s nullable references feature, there's a good chance that the first pain point will involve serialization. For example, if you use JSON.NET you might start seeing CS8618 warnings complaining that some non-nullable property is uninitialized. In this post I'll explain what's going on and what you can do about it.

The underlying problem is simple. We've enabled nullable reference annotations and warnings by adding <Nullable>enable</Nullable> to a property group in our project file, which has two important effects here. First, if we use a reference type (such as string), then unless we state that it may be null by appending a ?, the compiler will interpret that to mean that we never want the relevant variable, field, or property to be null. Second, the compiler will warn us if we write code that does not take steps to ensure that the value is in fact never null. Take this very simple example:

public class JsonSerializableType
{
    public int FavouriteInteger { get; set; }

    public string FavouriteColour { get; set; }
}

The first property here is an int, and it's simply not possible for this to have a null value. It defaults to 0, so even if we do nothing, its value will not be null. But the second property is of type string, a reference type, and it will default to null. If this code is in an enabled nullable annotation context, we are effectively telling the compiler that we want the property never to be null. And if the code is in an enabled nullable warning context, the compiler will report that we have failed to ensure non-nullness for this property, and so we get a CS8618.

The problem is that this class has no declared constructors, and although the compiler supplies a default constructor in these cases, that default constructor cannot usefully initialize the backing fields for these properties to anything other than their default values of 0 and null. This means that it's entirely possible for instances of this type to fail to honour their promise always to return non-null references from this property.

This can be very frustrating in cases where you know perfectly well that a non-null value will invariably be supplied immediately after construction. Perhaps you only ever create instances of this type with an object initializer in which you supply a non-null value. Or, maybe you only ever create instances of this type through deserialization—loading values out of JSON perhaps, or using the Microsoft.Extensions.Configuration.Binding feature that enables you to load values out of configuration settings into objects. How can you avoid these warnings in these cases?

Make nullable if appropriate

The most direct response to a CS8618 warning is to modify the relevant property to be nullable. In this case that would mean defining FavouriteColour as string? instead of string. In cases where null values really are expected this is exactly the right thing to do. You will be making it clear to consumers of your class that they cannot count on getting a non-null reference, so they will need to check and have some sort of fallback plan in cases where no string is present.

However, if a null value makes no sense, then it's a bad idea for your class to state otherwise. It might make the CS8618 warning go away, but you'll then get CS8602 warnings saying "Defererence of a possibly null reference" in code that tries to use the property. You're just pushing the problem further out instead of fixing it.

So you should only declare something to be nullable if it really is. For example, if your application needs to be able to cope with the possibility that someone simply doesn't have a favourite colour, then it declaring the property as nullable conveys that fact, and any CS8602 warnings that emerge as a result of attempts to use the property without checking for null are likely to indicate code that needs to be fixed.

Correct by construction

If a reference-typed property is naturally non-nullable, then the best way to avoid warnings is to require all non-nullable property values to be supplied during construction. In fact, you might want to consider requiring values for value-typed properties too. Just because it's impossible for an int to be null, that doesn't necessarily mean that it's always OK to accept the default of 0. It might be better to decide that unless you know the correct value for each property, you have no business constructing an object of this type:

public class JsonSerializableType
{
    public JsonSerializableType(
        int favouriteInteger, string favouriteColour)
    {
        FavouriteInteger = favouriteInteger;
        FavouriteColour = favouriteColour;
    }

    public int FavouriteInteger { get; set; }

    public string FavouriteColour { get; set; }
}

This banishes warnings. It's no longer possible to create an instance of this type without convincing the compiler that you've provided a non-null value for the non-nullable FavouriteColour property, so it'll won't show a CS8618 warning. But unlike the "fix" of adding a ? to keep the compiler happy, with this approach the property still correctly advertises its non-nullness, so you won't get CS8602 warnings in code that uses it. This approach is sometimes described as being "correct by construction."

A benefit of this approach is that it's easy to make the type immutable—if all properties are fully initialized by the constructor, you can drop the set accessors to make them all read-only. Immutability can rule out whole classes of errors. But that's just an optional bonus, not strictly related to nullable reference support.

But what about deserialization, you might be wondering? If I remove the default constructor, how is something like JSON.NET going to create an instance of this type? As it happens, the answer is: perfectly well. JSON.NET recognizes a pattern in which a type offers a constructor with parameters that correspond to properties. In this case it will understand that the int favouriteInteger parameter will initialize the int FavouriteInteger property, and likewise with the string favouriteColour parameter and the string FavouriteColour property. (And this works whether the relevant properties are read/write or read-only.)

If you've been following this series, you'll know that the nullable references feature cannot make guarantees—your code may well be used by non-nullable-reference-aware code, meaning that it's quite possible that your constructor may be invoked with a null reference despite the constructor signature declaring that this is not allowed. So in practice, you might want to check for nullness, e.g.:

public JsonSerializableType(int favouriteInteger, string favouriteColour)
{
    FavouriteInteger = favouriteInteger;
    FavouriteColour = favouriteColour
        ?? throw new ArgumentNullException(nameof(favouriteColour));
}

This is a pretty good solution. It prevents your code from accidentally constructing an object with a null where non-null is required. You'll get a compile time warning if you make that mistake from code in an enabled nullable warning context, and an exception at runtime if you try to do this from non-nullable-aware code. (At least, you will if you make the mistake during construction. Unfortunately if you also want property setters to throw exceptions, there's no way to automate that. That's another reason to make your objects read-only—if the only opportunity to provide a property value is during construction, thats the only place you ever need to check for null.) If you use this in conjunction with a serialization mechanism such as JSON.NET, you can be confident that if deserialization succeeds, all properties declared as non-nullable will be populated.

What's not to like?

Well there is one fly in the ointment. Not everything supports this idiom. For example, at endjin we make extensive use of the ‘binding' feature of the Microsoft.Extensions.Configuration libraries—this enables you to define a type that describes expected sets of configuration settings (to be set, e.g., through a JSON configuration file, or as application settings in an Azure App Service) as properties. This is convenient, and offers an improvement over code such as config["Preferences.FavouriteColour"] because it provides one place in which all the expected properties and their types are defined.

The one problem with Microsoft.Extensions.Configuration.Binding today (with version 3.1.6 of the library) is that it does not support this correct-by-construction idiom. It requires a zero-parameter constructor, and will throw an exception if one is not present.

What on earth are you supposed to do in this case?

Nullable backing field, non-nullable property

There is a pattern you can use in cases where you're obliged to provide a constructor that takes no arguments. You can declare a property as non-nullable, but use a nullable backing field for it, and then throw exceptions to ensure that it honours its assertion of non-nullness:

private string? favouriteColour;
public string FavouriteColour
{
    get => favouriteColour ?? throw new InvalidOperationException(
        $"nameof(FavouriteColour) must be assigned a non-null value before being read");
    set { favouriteColour = value ?? throw new ArgumentNullException(); }
}

This is less satisfactory than the correct-by-construction approach on two counts. First, this is much more verbose. Second, this means that errors are discovered later in the day. Since this technique relies on allowing an object to be in an invalid state after construction, we can only learn about a problem when something attempts to use an uninitialized property. In cases where properties are only used in specific scenarios, this can allow problems to lie undiscovered for some time. (E.g., you might make a change to configuration on a server that accidentally removes or misspells a property, and it will initially look fine, but may then blow up later when the property comes into play.)

However, it does have the merit of working. It enables your properties to declare their nullability correctly, and to ensure that they will never accept or return values that contradict the declared nullability, while still being able to use libraries that require a zero-parameter constructor.

But if this seems like too much work, you have another option

Selectively disable nullable reference support

If you have reason to be confident that you know something the compiler doesn't, and that a particular property will never be null in practice, you might simply want to tell the compiler to stop complaining. There is a #nullable directive that lets you change the nullable contexts within a single source file.

The first question is: what exactly should you disable? Should you move the relevant properties into a disabled nullable annotation context or a disabled nullable warning context? The latter will usually be the right answer in this scenario: you still want to be able to declare which properties can be expected to be null so that code that uses your type will get proper compiler messages regarding nullness. This means that you want them to be in an enabled nullable annotation context. Putting them in a disabled nullable warning context enables you to keep the annotation information but to avoid the warnings resulting from the compiler not knowing what you know:

#nullable disable warnings
    public string FavouriteColour { get; set; }
#nullable restore warnings

Of course, you'd better be right. We get no checks at all with this approach—the goal with this directive was to disable compile-time checking, but we've got no runtime checking either. If we are correct in thinking that we know better than the compiler, then that's fine, but if we're wrong, our property definitions are now writing cheques that our instances might not be able to cash.

The safest use of this technique is in cases where you are in fact doing some sort of check that guarantees non-nullness, and it's just that the compiler can't see these checks. (E.g., in the case where you're loading configuration settings, you could conceivably write a utility function which checks that all properties declared as non-null are non-null, and to apply this check directly after loading the configuration data and before your application starts to run.)

One gotcha with the preceding example is that it only works if you let the compiler generate a default constructor. If for some reason you need to add an explicit zero-arguments constructor (because you need to do some extra work that the compiler-generated default constructor would not) you'll find that this technique no longer works, because the CS8618 warning moves from the property to the constructor. In fact, a CS8618 only really ever applies to the constructor—the problem it describes occurs when it's possible for a constructor to return without having initialized a non-nullable field or auto-property. However, in cases where you don't write an explicit constructor, the compiler has to find something else to attach the warning to, so it puts it on the relevant property instead.

So as soon as you write any constructors of your own, the any CS8618 warnings will appear against the constructors, not the properties. You can just move the pragmas around the constructor instead, but it's a good deal less selective: instead of being able to opt out of warnings relating to one particular property, you are now disabling all possible CS8618 warnings within that constructor.

This problem is one argument in favour of disabling the nullable annotation context instead of the warning context, like so:

#nullable disable annotations
    public string FavouriteColour { get; set; }
#nullable restore annotations

With that in place, it no longer matters whether you let the compiler generate a default constructor, or you write your own: in either case the compiler will not complain if a constructor fails to initialize this property. In this case, it's not because we've supressed the relevant warning; it's because we've arranged for the warning not to apply. With these directives in place, the property and its backing field are both null-oblivious—they essentially live in the pre-nullable-references C# type system. This means it's not an error to leave them uninitialized. It also means that code that attempts to deference this property without first checking for null will not generate warnings. (That said, if you attempt to use the property where non-nullability is required, e.g., to initialize a non-nullable variable in an enabled nullable warning context, then you will get a warning.)

Will C# 9 record types help?

As I write this, C# 8.0 is the current version of C#. However, Microsoft has announced proposed features for C# 9, including something called record types. If you have seen these, you might have thought that these would provide an alternative solution to this problem. (I was expecting them to. A Roslyn developer at Microsoft describes this expectation as "the number 1 misconception" about this feature.) But the way they work in the previews available as I write this, they do not, and currently there are no plans to change this before the feature ships.

The basic concept behind record types is this: classes that are just a bunch of properties are a special case that the compiler can and should provide help with. The feature is still evolving, but the part of the current design that is relevant to this discussion introduces the idea of properties that are settable only during initialization, where "initialization" mean more than just construction: when creating an object from code, initialization includes setting properties with the object initializer syntax. Using the nightly build of the compiler available at the point where I wrote this, you would change our class by replacing each set with init:

// Note: syntax correct at start of August 2020, but this
// language feature has not yet shipped, so this could change.
public record JsonSerializableType
{
    public int FavouriteInteger { get; init; }

    public string FavouriteColour { get; init; }
}

This enables you to use object initializer syntax, e.g. new JsonSerializableType { FavouriteInteger = 42, FavouriteColour = "Blue" } . However, these properties cannot then be modified later on—they are settable only during initialization. (In fact, you can use this new init keyword without declaring the type as a record; the record keyword enables some additional class-level features.)

Unfortunately, there is currently no way to indicate that a property is required during initialization. This class definition still produces the same CS8618 warning.

There is a Required Properties feature proposal https://github.com/dotnet/csharplang/issues/3630 to address this, but it is currently labelled as a possible candidate for C# 10, so don't hold your breath.

Conclusion

If nullability accurately reflects your domain requirements, then by all means declare properties as nullable. Otherwise, where possible, enforce non-nullness by defining constructors that require values for all relevant properties. If that is unworkable, consider explicit property implementations that enforce nullness at the point of use—better late than never. But if the value this provides does not justify the efforts, consider using the #nullable directive to disable the warning, or, if all else fails, to make the property null-oblivious.