Blog

A new feature in C# 12.0 enables data types to define fixed-size arrays that don't require separate array objects on the heap. Learn how this is useful in performance-oriented and interop scenarios.

Fix the following issues: Errors converting large datasets to pandas, pandas for Spark is very slow, and pandas for Spark column reduction doesn't reduce data.

We benchmarked endjin's JSON Schema library on .NET 9.0 and saw large performance gains. There are even more gains to be had with new System.Text.Json features.

.NET 9.0 has shipped, and for the fourth year running, we benchmarked endjin's AIS.NET library and were very happy to see substantial performance gains, with no extra work required.

Extending carbon-optimised pipelines: choose between Azure regions at runtime, work around Wait activity limits, and adapt the pattern beyond the UK.

A portable Data Factory, Synapse, or Fabric pipeline that calls the Carbon Intensity API and waits for the greenest scheduling window — no custom code.

We have a wide range of computational mechanisms at our disposal, some of which emerged thanks to recent advances in AI. In this post, we look at the kinds of workloads that can take advantage of these.

C# 12.0's most prominent new feature is the primary constructor syntax. This post describes how it works, and looks at some pros and cons.

Translating carbon-optimised scheduling into a modern data pipeline architecture for Microsoft Fabric, Azure Synapse and Azure Data Factory.

Cloud data pipelines often have flexibility in when they run. Using the UK National Grid Carbon Intensity API, you can schedule them for the greenest window.

Thanks in part to recent advances in AI, we have a range of computational mechanisms at our disposal. However, certain universal truths apply to all of them.

Learn how to configure VS Code to enable stepping into external code when debugging a Python Behave test

C# 11.0 expanded high-performance, low-allocation functionality. This post describes the importance of the added support for ref fields, and how the scoped keyword relates.

AI's current hype cycle has driven massive hardware investment. Even if AI disappoints, those hardware capabilities remain. What else can we do with them?

Power BI's new PBIR format enhances collaboration, version control, and efficiency for developers. Learn key benefits and future implications.

Power BI Projects are a game changer for teams building reports; offering a source-control friendly format, CI/CD support, and the ability to edit in a code editor.

Spark doesn't always interpret JSON how we'd like. For example, if each key/value pair in a JSON object is conceptually one item, Spark won't give you a row per item by default. This article shows how to nudge Spark in the right direction.

This post examines .NET's native support for iterators: IEnumerator<T>, IEnumerable<T>, and IAsyncEnumerable<T>.

High-performance data teams start with C-suite sponsorship, align to organisational goals, adopt a product mindset, and position themselves as innovation engines rather than order takers.

Use Reactive Extensions for .NET to transform FileSystemWatcher events into a powerful and easy-to-use event stream to deal with file system idiosyncrasies.

Convert between unrelated IJsonValue types across API, CRM, and database schemas with near-zero allocation. A schema-first alternative to AutoMapper for JSON-driven .NET applications.

Set unevaluatedProperties to a schema to turn a JSON object into a Dictionary-like map. The generated C# type implements IReadOnlyDictionary with zero-allocation key comparisons.

Plain numeric enums lose their labels in JSON Schema. Using oneOf with const and title/description preserves named, documented numeric values with full pattern-matching support in C#.

JSON Schema's enum keyword constrains values to a fixed set of strings (or mixed types). Corvus.JsonSchema generates strongly typed constants with pattern-matching support in C#.

Use const-valued properties as discriminators in a oneOf union to implement the same polymorphic dispatch pattern used by System.Text.Json, illustrated with JSON Patch operations.