LINQ Max and nullable value types
While working on a project for a customer, we came across a slight oddity of LINQ's Max operator when you use it with a value type. In some cases Max returns null when supplied with an empty list, and there are cases where this works even with value types—this overload returns int? for example. But in some cases it will not do this, and will instead throw an exception if its input is empty. The reasons behind it are non-obvious and somewhat subtle, so I thought I'd write about it.
The Max operator offers a projection-based overload with this signature:
public static TResult? Max<TSource,TResult>(
this IEnumerable<TSource> source,
Func<TSource,TResult> selector);
This will iterate through the source, pass each item to the selector callback, and then return the highest of the values the callback returns.
Notice how although the selector function returns a TResult, the return type of Max itself is TResult?. That nullability is there to handle the case where the source enumerable is empty: in that case there is no maximum value (because there are no values at all) and that TResult? return type means Max can return null to indicate that.
But it goes a bit weird if the selector returns a value type. Suppose you've got this type (which is a reference type, but crucially, two of its properties use value types):
public record WithValues(string Label, int Number, DateTimeOffset Date);
First, let's verify that Max does what I've said with an empty list when the projection retrieves a reference type:
WithValues[] empty = [];
string? maxLabel = empty.Max(x => x.Label);
Console.WriteLine(maxLabel is null);
This prints out True, confirming that Max here does indeed return null to let us know that there was no maximum value. (The notion of a maximum string value raises the awkward fact that Max doesn't let you pass an IComparer<T> here, but let's ignore that for now.)
With that in mind, what type do you suppose maxDate has in this example?
WithValues[] empty = [];
var maxDate = empty.Max(d => d.Date);
If you look at the definition of Max you could correctly conclude that TSource here becomes WithValues and that TResult is DateTimeOffset. (And as we're doing all this type inference in our heads, we might reflect on whether using var here has really saved us any time and effort.) And since Max returns TResult? you might conclude that maxDate must be of type DateTimeOffset? (which is an alias for Nullable<DateTimeOffset>).
But that would be wrong. Here's exactly equivalent code using an explicit type declaration instead of var:
WithValues[] empty = [];
DateTimeOffset maxDate = empty.Max(d => d.Date);
It is now clear that maxDate's type is DateTimeOffset. If we were to try to declare it as a DateTimeOffset?, that would actually compile, but it wouldn't be equivalent to the var example: in the case where we use var, maxDate really does have the non-nullable DateTimeOffset type.
And if we do try to use DateTimeOffset?, it goes wrong. This compiles:
DateTimeOffset? maxDate = empty.Max(d => d.Date);
if (maxDate.HasValue)
{
Console.WriteLine(maxDate.Value);
}
else
{
Console.WriteLine("No dates found.");
}
but it only compiles without error because an implicit conversion is available from Max's return type of DateTimeOffset to the variable's type of DateTimeOffset?.
The most important thing to know about this code is that it will actually fail at runtime with an InvalidOperationException complaining that the Sequence contains no elements!
Earlier I linked to a non-generic overload of Max that returns an int? so you might think that this would work:
WithValues[] empty = [];
int? maxNumber = empty.Max(x => x.Number);
but this will also fail with an exception at runtime instead of returning null. In fact it ends up using a different overload that returns an int, and not the one that returns an int?.
So that's weird. The first two examples call the same single overload of Max, and yet it handles an empty list completely differently depending on whether our selector returns the Label or Date. (When it returns Number, we end up using the int-specific overload, but the fact remains that an empty list causes an exception when we select a value-typed property, but the method returns null when selecting a reference-typed property.) What's going on?
Well it turns out that this particular Max method actually has two different code paths: and it effectively uses this test to choose which path to use:
TResult val = default;
if (val == null)
...
If val == null, then it goes down the code path that returns null if the list is empty. If not, it goes down the path that throws an exception if the list is empty.
This is a deliberate design choice. If default(TResult) is something other than null—e.g. default(int) is 0—then there might be no way to tell the difference between an empty list, and a list where default(TResult) really was the maximum value. For example, in the list [-3,-2,-1,0], the maximum value is 0, so how could we distinguish between that case and the empty list case if we were getting back 0 in either case?
So there's a rationale for this behaviour, but it's not obvious that this one method can behave in two quite different ways. The documentation doesn't mention that this particular overload may throw an InvalidOperationException.
We can explore what that test will do with various types:
static void ShowNull<T>()
{
T? val = default;
Console.WriteLine(val == null);
}
ShowNull<string>();
ShowNull<string?>();
ShowNull<int>();
ShowNull<DateTimeOffset>();
ShowNull<int?>();
ShowNull<DateTimeOffset?>();
This prints out:
True
True
False
False
True
True
So this tells us that Max will consider TResult to be potentially nullable if it's a reference type like string, or if it's a nullable value type like int? or DateTimeOffset?. But plain value types like int and DateTimeOffset are considered not to be nullable.
That explains why using the x => x.Label lambda makes Max return null when the list is empty, while with d => d.Date or d => d.Number, it throws an exception. The first has a return type of string (a reference type) while the other two have non-nullable value-typed return types (DateTimeOffset and int).
But why does Max even have these two different code paths? It's perfectly possible for a method to return a DateTimeOffset?, so why does Max not do that here? If the argument for the type parameter TResult is DateTimeOffset, and the method declares a return type of TResult?, shouldn't that make the return type DateTimeOffset??
The reason it doesn't work out that way is because nullability handling for reference types was a bit of an afterthought in C#. (See my extensive series on nullable reference types for (a lot) more detail.)
In the beginning (C# 1.0) there were value types, which could not be null, and reference types, which were always capable of being null. There simply wasn't any concept of a value type being nullable, and nor was there any way to constrain a reference type to be non-null. This reflected the underlying reality of the .NET runtime's type system. Then C# 2.0 added support for nullable value types, enabling us to write int?. But this was an entirely different way of representing nullability: an int? is really a Nullable<int>, and Nullable<T> essentially combines a value with a bool indicating whether the value is present. This is fundamentally different from how reference types like string represent null. (This is more of a library feature than a runtime feature. OK, strictly speaking there's some special handling for Nullable<T> when it comes to boxing and unboxing, but otherwise, this is mainly a language feature that doesn't directly reflect how the underlying runtime type system really works.) Although C# lets us write code that works with int? in ways that are (sometimes) similar to how we might work with a reference, the generated code is really quite different, and that causes challenges for generic code. And finally, C# 8.0 introduced nullability annotations for reference types, so that now, we write string? if we mean a reference that might be null whereas string is (in theory) never null, in a way that is conceptually similar to the fact that an int can never be null.
But although we've ended up in a place where there are apparently two dimensions—value vs reference, and nullable vs non-nullable—the history of how we got here means these aren't truly independent. Nullability works very differently for values vs references in practice. And two of the four combinations (nullable values, and non-nullable references) aren't really first class citizens in the .NET type system. (A nullable value in a null state looks different from the null reference value. And a 'non-nullable' reference might in fact be null.)
And this difference tends to poke out from time to time with surprising behaviour like we're seeing with this Max operator. It would be completely reasonable to expect it to deal with the Label and Date properties in exactly the same way. But the history of nullability in .NET means it doesn't work in practice.
So how do we fix this? We can use this slightly ugly hack:
DateTimeOffset? maxDate = empty.Max(d => (DateTimeOffset?)d.Date);
That cast means that the lambda's type is now Func<WithValues, DateTimeOffset?>. (Before it was Func<WithValues, DateTimeOffset>, with a non-nullable return type.) Since default(DateTimeOffset?) == null, Max will select the code path that returns null when the input collection is empty. (It doesn't do that without this cast, because default(DateTimeOffset) is not null. It's a value representing midnight on the 1st January in the year 1, with a zero time zone offset.)
But what about that specialized (non-generic) overload of Max I linked to earlier that returns an int?? Well it turns out that it only comes into play when the selector also returns an int?. You get a different overload when the selector returns a plain int. So it ends up looking similar to the DateTimeOffset case (which used the generic overload). We need to cast to a nullable value:
int? maxNumber = empty.Max(x => (int?)x.Number);
So we can make it work how we want, it's just slightly messy. That's the reality of a 25+ year old language that has made two major changes to the nature of what it means to be null.
