Editions

For the first time since 2020, a new release of Rx.NET is available, supporting .NET 6 and .NET 7.

C# 11.0 adds some new pattern matching features. In this post, Ian shows the new support for matching lists.

Enabling 2FA is a must, but can often be perceived as a blocker for UI automation testing. Here's how to configure your AAD or MSA 2FA profile so that you can automate auth with a Time-based One-Time Password

Microsoft Fabric extends the promise of Azure Synapse integration to all analytics workloads from the data engineer to the business knowledge worker. It brings together Power BI, Data Factory, and the Data Lake, on a new generation of the Synapse data infrastructure. Delivered as a unified SaaS offering, it aims to reduce cost and time to value, while enabling new "citizen data science" capabilities. Check out all the resources from the endjin team collated in this post.

OneLake is the core of Microsoft Fabric - it is essentially the storage account for all of your data utilized within Fabric, be that within Azure or from another cloud (through Shortcuts). As the name suggests, it is a single, logical data lake backing more or less all of your Fabric workloads. For fans of analogies: you'll have probably heard Microsoft use the tag-line "OneLake is to data what OneDrive is to files/documents". Read this blog to dive deeper into OneLake and learn about why you should be excited.

Microsoft Fabric is a unified data & analytics platform that brings together, and improves upon, Microsoft's existing suite of data products. It can be thought of as the third generation of Microsoft data platforms, where first generation (e.g. HDInsight, SQL Data Warehouse) were somewhat isolated takes on traditional data products; second generation was Azure Synapse Analytics, which integrated platforms at a UX level but still felt a little disjointed at the data level; and now we have Microsoft Fabric which builds upon the Synapse "unification" vision, with a particular focus on enabling deep data-level interoperability. In this post we introduce its core offerings, as well as weigh up the pros and cons of this new product.

Charlotte is studying for a Bachelor of Engineering - BEng (Hons), Computer Science at the University of York. She was part of our 2021 internship cohort, and is spending her Year in Industry placement with endjin too. In this post she reflects on her first 6 months.

Jessica joined endjin as part of the Software Engineering Apprenticeship 2021 cohort. In this post, Jessica reflects on her first year.

Liam joined endjin as part of the Software Engineering Apprenticeship 2021 cohort. In this post he looks back on his fist year and uses the post to introspect and look ahead to his second year.

Performing certain deployment operations from a cloud-based CI/CD agent against resources that are only accessible via private networking are problematic. The often-cited solution is to use your own hosted agents that are also connected to the same private networks, however, this introduces additional costs and maintenance overhead. This post discusses an approach that combines the use of cloud-based CI/CD agents with 'just-in-time' allow-listing as an alternative to operating your own private agents.

Why you should make writing unit tests a regular practice and how they will improve your code. This blog explains how to write unit tests in PowerShell using Pester.

If you're interested in the Distributed Application Runtime (Dapr) and how it can simplify migrating existing workloads to use cloud platform services, then this series might be for you. In this post, we use the container tools in Visual Studio to debug why a newly-containerised service is not working.

Enabling 2FA is a must, but can often be perceived as a blocker for UI automation testing. Here's how to configure your AAD or MSA 2FA profile so that you can automate auth with a Time-based One-Time Password

Using multiple versions of Python on Windows can be a frustrating experience, especially if you've experienced how smooth the same workflow is on MacOS and Linux/WSL. This post provides a script and guide to getting setup to use Python virtual environments and Poetry, a popular dependency management tool.

This series will follow our ongoing journey of migrating the CI/CD pipelines for our many Open Source repositories, from Azure DevOps to GitHub Actions.

Microsoft Fabric extends the promise of Azure Synapse integration to all analytics workloads from the data engineer to the business knowledge worker. It brings together Power BI, Data Factory, and the Data Lake, on a new generation of the Synapse data infrastructure. Delivered as a unified SaaS offering, it aims to reduce cost and time to value, while enabling new "citizen data science" capabilities. Check out all the resources from the endjin team collated in this post.

OneLake is the core of Microsoft Fabric - it is essentially the storage account for all of your data utilized within Fabric, be that within Azure or from another cloud (through Shortcuts). As the name suggests, it is a single, logical data lake backing more or less all of your Fabric workloads. For fans of analogies: you'll have probably heard Microsoft use the tag-line "OneLake is to data what OneDrive is to files/documents". Read this blog to dive deeper into OneLake and learn about why you should be excited.

Microsoft Fabric is a unified data & analytics platform that brings together, and improves upon, Microsoft's existing suite of data products. It can be thought of as the third generation of Microsoft data platforms, where first generation (e.g. HDInsight, SQL Data Warehouse) were somewhat isolated takes on traditional data products; second generation was Azure Synapse Analytics, which integrated platforms at a UX level but still felt a little disjointed at the data level; and now we have Microsoft Fabric which builds upon the Synapse "unification" vision, with a particular focus on enabling deep data-level interoperability. In this post we introduce its core offerings, as well as weigh up the pros and cons of this new product.

Microsoft Fabric extends the promise of Azure Synapse integration to all analytics workloads from the data engineer to the business knowledge worker. It brings together Power BI, Data Factory, and the Data Lake, on a new generation of the Synapse data infrastructure. Delivered as a unified SaaS offering, it aims to reduce cost and time to value, while enabling new "citizen data science" capabilities. Check out all the resources from the endjin team collated in this post.

OneLake is the core of Microsoft Fabric - it is essentially the storage account for all of your data utilized within Fabric, be that within Azure or from another cloud (through Shortcuts). As the name suggests, it is a single, logical data lake backing more or less all of your Fabric workloads. For fans of analogies: you'll have probably heard Microsoft use the tag-line "OneLake is to data what OneDrive is to files/documents". Read this blog to dive deeper into OneLake and learn about why you should be excited.

In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?

There is a new service in town that promises to transform the way you query the contents of your data lake. Azure Synapse Analytics comes with a new offering called SQL Serverless allowing you to query your data on-demand with no need for pre-provisioned resources.When we heard about the new service we were keen to get involved, so for the last 10 months we've been working with the SQL Serverless product group to provide feedback on the service and to help ensure it meets our customers needs. During this time we've put it through it's paces by implementing a range of real-world use cases. We were particularly interested to see how it stacked up as a replacement for Data Lake Analytics, where to date there has been no clear and easy migration path.

Have you or are you about to invest in Azure Databricks? If so, the new Spark offering in Azure Synapse Analytics is likely to have grabbed your attention and rightly so. Why is Microsoft putting yet another Spark offering on the table and what does it mean for you?

Microsoft Fabric extends the promise of Azure Synapse integration to all analytics workloads from the data engineer to the business knowledge worker. It brings together Power BI, Data Factory, and the Data Lake, on a new generation of the Synapse data infrastructure. Delivered as a unified SaaS offering, it aims to reduce cost and time to value, while enabling new "citizen data science" capabilities. Check out all the resources from the endjin team collated in this post.

OneLake is the core of Microsoft Fabric - it is essentially the storage account for all of your data utilized within Fabric, be that within Azure or from another cloud (through Shortcuts). As the name suggests, it is a single, logical data lake backing more or less all of your Fabric workloads. For fans of analogies: you'll have probably heard Microsoft use the tag-line "OneLake is to data what OneDrive is to files/documents". Read this blog to dive deeper into OneLake and learn about why you should be excited.

Microsoft Fabric is a unified data & analytics platform that brings together, and improves upon, Microsoft's existing suite of data products. It can be thought of as the third generation of Microsoft data platforms, where first generation (e.g. HDInsight, SQL Data Warehouse) were somewhat isolated takes on traditional data products; second generation was Azure Synapse Analytics, which integrated platforms at a UX level but still felt a little disjointed at the data level; and now we have Microsoft Fabric which builds upon the Synapse "unification" vision, with a particular focus on enabling deep data-level interoperability. In this post we introduce its core offerings, as well as weigh up the pros and cons of this new product.

In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.

Do you struggle to manage conflicting demands from multiple stakeholders, even when they have agreed about the relative priority of their requests? Apply a budgeting methodology to agile projects for better planning and communication with the team and wider stakeholders.

Learn about the thinking behind endjin's Power BI Maturity assessment by applying Wardley Doctrine, and asking more questions.

Jessica joined endjin as part of the Software Engineering Apprenticeship 2021 cohort. In this post, Jessica reflects on her first year.

In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.

Data validation is a vital step in any data-oriented workstream. This post investigates and compares two popular Python data validation packages - Pandera and Great Expectations

Using multiple versions of Python on Windows can be a frustrating experience, especially if you've experienced how smooth the same workflow is on MacOS and Linux/WSL. This post provides a script and guide to getting setup to use Python virtual environments and Poetry, a popular dependency management tool.

Microsoft Fabric is a unified data & analytics platform that brings together, and improves upon, Microsoft's existing suite of data products. It can be thought of as the third generation of Microsoft data platforms, where first generation (e.g. HDInsight, SQL Data Warehouse) were somewhat isolated takes on traditional data products; second generation was Azure Synapse Analytics, which integrated platforms at a UX level but still felt a little disjointed at the data level; and now we have Microsoft Fabric which builds upon the Synapse "unification" vision, with a particular focus on enabling deep data-level interoperability. In this post we introduce its core offerings, as well as weigh up the pros and cons of this new product.

Databricks notebooks provide an interactive and collaborative environment for developing code while applying software engineering best practices, such as source control. This blog post provides a step by step guide of how to implement source control in Databricks.

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

Open-source software has become an essential part of many organisation's software supply chain, however, this poses challenges with license compliance and security assurance.

Data validation is a vital step in any data-oriented workstream. This post investigates and compares two popular Python data validation packages - Pandera and Great Expectations

Using multiple versions of Python on Windows can be a frustrating experience, especially if you've experienced how smooth the same workflow is on MacOS and Linux/WSL. This post provides a script and guide to getting setup to use Python virtual environments and Poetry, a popular dependency management tool.

Do you struggle to manage conflicting demands from multiple stakeholders, even when they have agreed about the relative priority of their requests? Apply a budgeting methodology to agile projects for better planning and communication with the team and wider stakeholders.

For the first time since 2020, a new release of Rx.NET is available, supporting .NET 6 and .NET 7.

Databricks notebooks provide an interactive and collaborative environment for developing code while applying software engineering best practices, such as source control. This blog post provides a step by step guide of how to implement source control in Databricks.

Monetizing your APIs is a complex challenge; creating your business, revenue, & pricing model, then translating into a technical implementation requires significant investment. Working with the Azure API Management team we provide guidance, videos and code to help accelerate your journey.

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

Tracking down unexpected allocations in a high-performance .NET parsing library.

This is a summary of the sessions I attended at SQLbits 2023 in Newport Wales, which is Europe's largest expert led data conference.

This is a summary of the sessions I attended at SQLbits 2022 in London, which is Europe's largest expert led data conference.

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Microsoft Fabric extends the promise of Azure Synapse integration to all analytics workloads from the data engineer to the business knowledge worker. It brings together Power BI, Data Factory, and the Data Lake, on a new generation of the Synapse data infrastructure. Delivered as a unified SaaS offering, it aims to reduce cost and time to value, while enabling new "citizen data science" capabilities. Check out all the resources from the endjin team collated in this post.

OneLake is the core of Microsoft Fabric - it is essentially the storage account for all of your data utilized within Fabric, be that within Azure or from another cloud (through Shortcuts). As the name suggests, it is a single, logical data lake backing more or less all of your Fabric workloads. For fans of analogies: you'll have probably heard Microsoft use the tag-line "OneLake is to data what OneDrive is to files/documents". Read this blog to dive deeper into OneLake and learn about why you should be excited.

In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.

For the first time since 2020, a new release of Rx.NET is available, supporting .NET 6 and .NET 7.

This blog shows how to transform a document based format to a tabular format whilst managing all the nested properties

When implementing OpenChain, understanding the specification will help guide your organisation to having processes in place to review and manage open-source software

Open-source software has become an essential part of many organisation's software supply chain, however, this poses challenges with license compliance and security assurance.

When implementing OpenChain, understanding the specification will help guide your organisation to having processes in place to review and manage open-source software

The key risks associated with open-source software, from whether you use it minimally, to using it throughout all your systems.

Microsoft Fabric extends the promise of Azure Synapse integration to all analytics workloads from the data engineer to the business knowledge worker. It brings together Power BI, Data Factory, and the Data Lake, on a new generation of the Synapse data infrastructure. Delivered as a unified SaaS offering, it aims to reduce cost and time to value, while enabling new "citizen data science" capabilities. Check out all the resources from the endjin team collated in this post.

Microsoft Fabric is a unified data & analytics platform that brings together, and improves upon, Microsoft's existing suite of data products. It can be thought of as the third generation of Microsoft data platforms, where first generation (e.g. HDInsight, SQL Data Warehouse) were somewhat isolated takes on traditional data products; second generation was Azure Synapse Analytics, which integrated platforms at a UX level but still felt a little disjointed at the data level; and now we have Microsoft Fabric which builds upon the Synapse "unification" vision, with a particular focus on enabling deep data-level interoperability. In this post we introduce its core offerings, as well as weigh up the pros and cons of this new product.

If the audience for your Power BI report is larger than 12 people, then statistically 1 of your audience suffers from Colour Vision Deficiency. In this post we will talk through how we experimented with colour to develop an accessible colour palette and why colour and contrast is important when it comes to data visualisation.

In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.

Data validation is a vital step in any data-oriented workstream. This post investigates and compares two popular Python data validation packages - Pandera and Great Expectations

Using multiple versions of Python on Windows can be a frustrating experience, especially if you've experienced how smooth the same workflow is on MacOS and Linux/WSL. This post provides a script and guide to getting setup to use Python virtual environments and Poetry, a popular dependency management tool.

Open-source software has become an essential part of many organisation's software supply chain, however, this poses challenges with license compliance and security assurance.

When implementing OpenChain, understanding the specification will help guide your organisation to having processes in place to review and manage open-source software

The key risks associated with open-source software, from whether you use it minimally, to using it throughout all your systems.

Monetizing your APIs is a complex challenge; creating your business, revenue, & pricing model, then translating into a technical implementation requires significant investment. Working with the Azure API Management team we provide guidance, videos and code to help accelerate your journey.

Microsoft recently shot a video interviewing endjin co-founder, Howard van Rooijen, and Director of Engineering, James Broome, about how Microsoft has helped endjin grow over the past decade. This posts the top 10 ways in which Microsoft helped - from providing access to valuable software and services, to opening up sales channels, to helping to navigate the minefield of UK Financial Services regulations around cloud adoption.

In this post we discuss the characteristics of a great FinTech startup, and the importance of the API Economy to innovation in Financial Services.

Do you struggle to manage conflicting demands from multiple stakeholders, even when they have agreed about the relative priority of their requests? Apply a budgeting methodology to agile projects for better planning and communication with the team and wider stakeholders.

Learn about the thinking behind endjin's Power BI Maturity assessment by applying Wardley Doctrine, and asking more questions.

This is a summary of the sessions I attended at SQLbits 2023 in Newport Wales, which is Europe's largest expert led data conference.

If the audience for your Power BI report is larger than 12 people, then statistically 1 of your audience suffers from Colour Vision Deficiency. In this post we will talk through how we experimented with colour to develop an accessible colour palette and why colour and contrast is important when it comes to data visualisation.

In this post we will talk through how to expand on a data team's creative skillset, without access to specialist photo editing software such as Photoshop or Illustrator.

Field Parameters allow users to dynamically change the measure being analysed within a report. This blog post will explore how to use the new Field Parameters feature to dynamically switch between measures in a visua.

If the audience for your Power BI report is larger than 12 people, then statistically 1 of your audience suffers from Colour Vision Deficiency. In this post we will talk through how we experimented with colour to develop an accessible colour palette and why colour and contrast is important when it comes to data visualisation.

In this post we will talk through how to expand on a data team's creative skillset, without access to specialist photo editing software such as Photoshop or Illustrator.

Field Parameters allow users to dynamically change the measure being analysed within a report. This blog post will explore how to use the new Field Parameters feature to dynamically switch between measures in a visua.