Skip to content

A Short Tale of a Deceptively Slow LINQ Expression

Howard van Rooijen By Howard van Rooijen Co-Founder
A Short Tale of a Deceptively Slow LINQ Expression

With any good iterative development process – the first step is to get something working: solve the problem in as simple a fashion as possible, then refactor. Part of the refactoring consideration should be an initial performance monitoring spike to see which parts of your codebase are running slow.

Once I had the initial prototype of Templify working I decided to performance profile it – as I was aware that the package creation and package deployment processes were taking quite a bit of time, not an exceedingly large amount of time – as it was doing considerable work: lots of IO tasks – scanning and copying folders and file, reading and modifying file contents, creating archive files etc.

One of the first performance hotspots highlighted was the following piece of code that create the Templify manifest – the list of files that make up the template:

My mantra for performance optimisation spikes is MAM: "Measure, Alter, Measure" – the above code took 213 seconds to run against my test harness (the Sharp Architecture 1.6 release). Initially I thought that 3.5 minutes wasn't an unreasonable amount of time – and the code is trivial – it's looping around a collection of files, doing some string manipulation, adding a result to another collection and raising an event. Thinking about it some more – there is no way that such trivial code should take that long – so there had to be something else going on.

Then I suddenly realised that it was a problem with the LINQ extensions – they were being evaluated on every iteration of the loop – firstly the outer foreach and then the inner files.Count(). I made one simple refactoring to force evaluation of the collection (rather than being deferred) and calculating the total number of items up front rather than repeatedly in an adhoc manner:

for-each-after

I measured the code running against my test harness for a second time – the method now executed in 84 seconds. Such a small change caused an overall saving of 129 seconds.

The Introduction to Rx.NET (v6.1) 3rd Edition (2025) Book, by Ian Griffiths & Lee Campbell, is now available to download for FREE.

The lesson to take away is that just because code is simple, doesn't mean it performs well. LINQ exposes simple extension points that mask deceptively complex inner workings that can have hidden performance penalties if used incorrectly.

Programming C# 12 Book, by Ian Griffiths, published by O'Reilly Media, is now available to buy.

For those who are interested - my tool of choice for Refactoring is ReSharper and my tool of choice for Performance Profiling is dotTrace.

FAQs

Why can simple-looking LINQ code have hidden performance problems? LINQ extensions use deferred execution, meaning they evaluate on every iteration of a loop. If you call methods like Count() inside a foreach loop over a LINQ query, the query is re-evaluated on every iteration. This can turn trivial code into a performance nightmare.
What is the MAM approach to performance optimisation? MAM stands for Measure, Alter, Measure. Before making changes, measure the current performance to establish a baseline. Then make a targeted alteration and measure again to verify the improvement. This approach ensures you fix actual problems rather than making assumptions.
How do you fix LINQ deferred execution performance issues? Force evaluation of the collection upfront by calling ToList() or ToArray() before the loop, and calculate aggregate values like Count() once before iterating. In this case, a single refactoring reduced execution time from 213 seconds to 84 seconds, saving 129 seconds.

Also worth reading:

A follow up to "A Short Tale of a Deceptively Slow LINQ Expression"

A follow up to "A Short Tale of a Deceptively Slow LINQ Expression"

Howard van Rooijen

My last post caused a number of responses from the community. I discuss them here and provide a little bit more context.

A Step by Step Guide to MongoDB for .NET Developers

Howard van Rooijen

MongoDB has been taken a large amount of mindshare in the NoSQL space. In this blog post we share a simple step-by-step guide on how to get started as a .NET developer.
Introducing Templify

Introducing Templify

Howard van Rooijen

Templify is a simple tool for tokenising and packaging entire code based solutions. It's designed to unfurl demos / quickstarts / solution templates.

Howard van Rooijen

Co-Founder

Howard van Rooijen

Howard spent 10 years as a technology consultant helping some of the UK's best known organisations work smarter, before founding endjin in 2010. He's a Microsoft ScaleUp Mentor, and a Microsoft MVP for Azure and Developer Technologies, and helps small teams achieve big things using data, AI and Microsoft Azure.