Blog

Explore DuckLake's advanced capabilities including built-in encryption, sophisticated conflict resolution, and the strategic implications for future data architecture. Understand how DuckLake enables new business models and positions itself against established lakehouse formats.

Get hands-on with DuckLake through a comprehensive tutorial covering installation, basic operations, file organization, snapshots, and time travel functionality. Learn how DuckLake's database-backed metadata management works in practice.

DuckDB Labs introduces DuckLake, a revolutionary approach to lakehouse architecture that solves fundamental problems with existing formats by bringing database principles back to data lake metadata management.

This post describes an approach to copy files and data from SharePoint into Azure using Synapse Pipelines.

The interactive notebook shared in this post defines the process of granting Service Principals (inc. Synapse managed identities) access to SharePoint sites.

.NET Aspire can create new local instances of services such as SQL Server from scratch each time you run. While this guarantees repeatability and isolation, it can be time consuming, so this post explores alternatives.

.NET Aspire can create all of the resources an integration test required, but the test will often need direct access to the same SQL database as the code under test. This post shows how to do that.

.NET Aspire can create local services such as a SQL Server to stand in for cloud resources under local development and testing. This post shows how to ensure such a database is suitably initialized.

.NET Aspire offers features enabling integration tests to use dev-time orchestration. This post shows how a test can use these.

.NET Aspire's dev-time orchestration can be used to build integration tests that depend on a SQL Server database. This blog series explains how.

The Medallion Architecture consists of three data tiers: Bronze (raw), Silver (clean), and Gold (projected). Data moves through these three tiers and becomes more opinionated at each stage.

In Paul Waller's final, and posthumously published blog post, he takes you through a creative walk-through of the Titanic Power BI Report he created with Barry Smart.

This is follow guide to designing a mobile navigation in Power BI, covering form, icons, states, actions, with a view to enhancing report design & UI.

Liam joined endjin as part of the Software Engineering Apprenticeship 2021 cohort. In this post he looks back on his time at endjin before moving on.

With new technologies - such as Delta Lake and other open table formats - there have been huge improvements the performance of Data Lakehouses. But what is Delta Lake and how does it work?

What exactly is a Data Lakehouse? This blog gives a general introduction to their history, functionality, and what they might mean for you!

Learn how to integrate DuckDB into enterprise environments, including Microsoft Fabric deployment, and explore the architectural patterns it enables for modern data processing workflows.

Dive deep into the technical details of DuckDB, exploring its columnar architecture, vectorized execution, SQL enhancements, and the performance optimizations that make it exceptionally fast on a single machine.

Explore the concept of the 'data singularity' and how in-process analytics tools like DuckDB are transforming how we work with data by leveraging modern hardware capabilities.

This blog explores how to implement robust validation strategies within the medallion architecture using Pandera, helping you catch issues early and maintain clean, trustworthy data.

Records are primarily meant for representing data. They are usually immutable and allow you to copy, equate, and print, object properties.

C# 12.0 adds a new way to annotate parameters: ref readonly. This seems like it should mean exactly the same as the older in annotation. This post explains why this new syntax is useful.

Understand why label encoding is the preferred technique for encoding categorical data for analysis in Power BI over one-hot encoding.

This post explains how covariance and contravariance in C# interfaces works

One-hot encoding and label encoding are two methods used to encode categorical data. Understand the specific advantages and disadvantages of these techniques.