Browse our archives by topic…
Editions
.NET Development
View all (318)
How .NET 8.0 boosted JSON Schema performance by 20%
We benchmarked endjin's JSON Schema library on .NET 8.0 and saw large performance gains. There are even more gains to be had with new .NET 8 features.

How .NET 8.0 boosted AIS.NET performance by 27%
When we benchmarked endjin's AIS.NET library on .NET 8.0, we were delighted to see substantial performance gains, with no extra work required.

Optimising .NET code: Avoiding allocations using Span<T>
Investigate some ways of working with strings that prevent unnecessary memory allocations.
Analytics
View all (127)
Notebooks in Azure Databricks
This blog post explores interactive notebooks in Azure Databricks. An Azure Databricks Notebook is a powerful data science tool that support exploratory data analysis, hypothesis testing, data cleaning and transformation, data visualisation, statistical modeling and machine learning.

Copilot - Are You Ready to Unleash the Power of AI in Self Service Analytics?
In the ever-evolving landscape of data and analytics, the advent of AI-powered capabilities has opened up exciting possibilities for self service reporting. Tools like Copilot in Power BI and Microsoft Fabric offer users the ability to extract insights from data using natural language prompts. It's an enticing prospect, where anyone can explore, visualize, and analyze data without being constrained by pre-canned reports or relying on data engineering teams. However, as we start to embrace these new capabilities, it's essential to strike a balance between the potential benefits and potential pitfalls.

Microsoft Fabric: Announced
Microsoft Fabric unifies Power BI, Data Factory & Data Lake on Synapse infrastructure, reducing cost & time while enabling citizen data science.
Apprenticeship
View all (57)
Life as an Apprentice Engineer at endjin
Eli joined endjin as part of the Software Engineering Apprenticeship 2021 cohort. In this post she reflects on her first two years.

My year in industry as a whole
As Charlotte's placement comes to an end, she reflects on her year at endjin, highlighting her experiences to take back to University with her

My Year in Industry so far
Charlotte is studying for a Bachelor of Engineering - BEng (Hons), Computer Science at the University of York. She was part of our 2021 internship cohort, and is spending her Year in Industry placement with endjin too. In this post she reflects on her first 6 months.
Architecture
View all (65)
Architecture Decision Records
Explore the benefits of Architecture Decision Records (ADRs) in technical design, with real-world examples applied to different scenarios.

Using Cloud CI/CD in Zero Trust Environments
Performing certain deployment operations from a cloud-based CI/CD agent against resources that are only accessible via private networking are problematic. The often-cited solution is to use your own hosted agents that are also connected to the same private networks, however, this introduces additional costs and maintenance overhead. This post discusses an approach that combines the use of cloud-based CI/CD agents with 'just-in-time' allow-listing as an alternative to operating your own private agents.

Creating Pester Unit Tests in PowerShell
Why you should make writing unit tests a regular practice and how they will improve your code. This blog explains how to write unit tests in PowerShell using Pester.
Automation
View all (4)
Using the Playwright C# SDK to automate 2FA authentication for AAD and MSA
Enabling 2FA is a must, but can often be perceived as a blocker for UI automation testing. Here's how to configure your AAD or MSA 2FA profile so that you can automate auth with a Time-based One-Time Password

How to setup Python, PyEnv & Poetry on Windows
Explore using Python virtual environments & Poetry on Windows for smoother workflows, with a script & guide to enhance your dependency management experience.

Adventures in GitHub Actions: Episode 1 - Goodbye Azure DevOps, Hello GitHub Actions
This series will follow our ongoing journey of migrating the CI/CD pipelines for our many Open Source repositories, from Azure DevOps to GitHub Actions.
Azure
View all (180)
Notebooks in Azure Databricks
This blog post explores interactive notebooks in Azure Databricks. An Azure Databricks Notebook is a powerful data science tool that support exploratory data analysis, hypothesis testing, data cleaning and transformation, data visualisation, statistical modeling and machine learning.

Microsoft Fabric: Announced
Microsoft Fabric unifies Power BI, Data Factory & Data Lake on Synapse infrastructure, reducing cost & time while enabling citizen data science.

What is OneLake?
Explore OneLake, Microsoft Fabric's core storage for data in Azure & other clouds. Discover its role in Fabric workloads, the OneDrive equivalent for data storage.
Azure Synapse Analytics
View all (37)
Star Schemas are fundamental to unleashing value from data in Microsoft Fabric
Ralph Kimble's 1996 principles of dimensional modelling for Star Schemas remain relevant in Cloud Native Analytics.

Adopt A Product Mindset To Maximise Value From Microsoft Fabric
In this post I describe how adopting a product mindset will help you to extract maximum value from Microsoft Fabric.

Exploring Strategies Enabled By Microsoft Fabric
Explore building situational awareness and leveraging strategic opportunities with Microsoft Fabric in this concise overview.
Big Compute
View all (27)
Azure Synapse Analytics: How serverless is replacing the data warehouse
Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?

Benchmarking Azure Synapse Analytics - SQL Serverless, using Polyglot Notebooks
New Azure Synapse Analytics service offers SQL Serverless for on-demand data lake queries. We tested its potential as a Data Lake Analytics replacement.

Does Azure Synapse Analytics spell the end for Azure Databricks?
Have you or are you about to invest in Azure Databricks? If so, the new Spark offering in Azure Synapse Analytics is likely to have grabbed your attention and rightly so. Why is Microsoft putting yet another Spark offering on the table and what does it mean for you?
Big Data
View all (91)
Adopt A Product Mindset To Maximise Value From Microsoft Fabric
In this post I describe how adopting a product mindset will help you to extract maximum value from Microsoft Fabric.

Exploring Strategies Enabled By Microsoft Fabric
Explore building situational awareness and leveraging strategic opportunities with Microsoft Fabric in this concise overview.

Developing a Data Mesh Inspired Vision Using Microsoft Fabric
Explore Microsoft Fabric, inspired by Data Mesh, for a data-driven strategy. Learn to approach a Data Mesh vision using this powerful tool.
Big DataServerless
View all (1)
Azure Synapse Analytics versus Microsoft Fabric: A Side by Side Comparison
In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.
Culture
View all (126)
Life as an Apprentice Engineer at endjin
Eli joined endjin as part of the Software Engineering Apprenticeship 2021 cohort. In this post she reflects on her first two years.

Agile teams often get their planning and priorities wrong
Do you struggle to manage conflicting demands from multiple stakeholders, even when they have agreed about the relative priority of their requests? Apply a budgeting methodology to agile projects for better planning and communication with the team and wider stakeholders.

Ask the right questions to get your data insights projects back on track
Learn about the thinking behind endjin's Power BI Maturity assessment by applying Wardley Doctrine, and asking more questions.
Data Engineering
View all (25)
Star Schemas are fundamental to unleashing value from data in Microsoft Fabric
Ralph Kimble's 1996 principles of dimensional modelling for Star Schemas remain relevant in Cloud Native Analytics.

Adopt A Product Mindset To Maximise Value From Microsoft Fabric
In this post I describe how adopting a product mindset will help you to extract maximum value from Microsoft Fabric.

Exploring Strategies Enabled By Microsoft Fabric
Explore building situational awareness and leveraging strategic opportunities with Microsoft Fabric in this concise overview.
Databricks
View all (8)
Intro to Microsoft Fabric
Microsoft Fabric unifies data & analytics, building on Azure Synapse Analytics for improved data-level interoperability. Explore its offerings & pros/cons.

Version Control in Databricks
Databricks notebooks provide an interactive and collaborative environment for developing code while applying software engineering best practices, such as source control. This blog post provides a step by step guide of how to implement source control in Databricks.

How to use SQL Notebooks to access Azure Synapse SQL Pools & SQL on demand
Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.
DevOps
View all (33)
Polyglot Notebooks for Ops
Polyglot Notebooks' PowerShell support enhances IT Ops with robust, repeatable processes via 'executable documentation'.

Exploring OpenChain: From License Compliance to Security Assurance
Open-source software has become an essential part of many organisation's software supply chain, however, this poses challenges with license compliance and security assurance.

Data validation in Python: a look into Pandera and Great Expectations
Implement Python data validation with Pandera & Great Expectations in this comparison of their features and use cases.
Engineering Practices
View all (142)
Architecture Decision Records
Explore the benefits of Architecture Decision Records (ADRs) in technical design, with real-world examples applied to different scenarios.

Agile teams often get their planning and priorities wrong
Do you struggle to manage conflicting demands from multiple stakeholders, even when they have agreed about the relative priority of their requests? Apply a budgeting methodology to agile projects for better planning and communication with the team and wider stakeholders.

Rx.NET v6.0 Now Available
For the first time since 2020, a new release of Rx.NET is available, supporting .NET 6 and .NET 7.
Innovation
View all (30)
How to Monetize APIs with Azure API Management
Monetizing your APIs is a complex challenge; creating your business, revenue, & pricing model, then translating into a technical implementation requires significant investment. Working with the Azure API Management team we provide guidance, videos and code to help accelerate your journey.

Do robots dream of counting sheep?
Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Azure Synapse Analytics: How serverless is replacing the data warehouse
Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?
Internet of Things
View all (15)
Do robots dream of counting sheep?
Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

How to use SQL Notebooks to access Azure Synapse SQL Pools & SQL on demand
Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

ArrayPool vs MemoryPool—minimizing allocations in AIS.NET
Tracking down unexpected allocations in a high-performance .NET parsing library.
Machine Learning
View all (31)
SQLbits 2023 - The Best Bits
This is a summary of the sessions I attended at SQLbits 2023 in Newport Wales, which is Europe's largest expert led data conference.

SQLbits 2022 - The Best Bits
This is a summary of the sessions I attended at SQLbits 2022 in London, which is Europe's largest expert led data conference.

Do robots dream of counting sheep?
Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?
Microsoft Fabric
View all (11)
Star Schemas are fundamental to unleashing value from data in Microsoft Fabric
Ralph Kimble's 1996 principles of dimensional modelling for Star Schemas remain relevant in Cloud Native Analytics.

Adopt A Product Mindset To Maximise Value From Microsoft Fabric
In this post I describe how adopting a product mindset will help you to extract maximum value from Microsoft Fabric.

Exploring Strategies Enabled By Microsoft Fabric
Explore building situational awareness and leveraging strategic opportunities with Microsoft Fabric in this concise overview.
Open Source
View all (62)
Implementing the OpenChain Specification
After a year of working on implementing the OpenChain specification, this blog takes you through the processes we created to track and manage our open-source licenses

Rx.NET v6.0 Now Available
For the first time since 2020, a new release of Rx.NET is available, supporting .NET 6 and .NET 7.

Working with JSON in Pyspark
This blog shows how to transform a document based format to a tabular format whilst managing all the nested properties
OpenChain
View all (4)
Exploring OpenChain: From License Compliance to Security Assurance
Open-source software has become an essential part of many organisation's software supply chain, however, this poses challenges with license compliance and security assurance.

The OpenChain specification explained
When implementing OpenChain, understanding the specification will help guide your organisation to having processes in place to review and manage open-source software

What are the risks with open-source software?
The key risks associated with open-source software, from whether you use it minimally, to using it throughout all your systems.
Power BI
View all (66)
Star Schemas are fundamental to unleashing value from data in Microsoft Fabric
Ralph Kimble's 1996 principles of dimensional modelling for Star Schemas remain relevant in Cloud Native Analytics.

Microsoft Fabric: Announced
Microsoft Fabric unifies Power BI, Data Factory & Data Lake on Synapse infrastructure, reducing cost & time while enabling citizen data science.

Intro to Microsoft Fabric
Microsoft Fabric unifies data & analytics, building on Azure Synapse Analytics for improved data-level interoperability. Explore its offerings & pros/cons.
Python
View all (16)
Azure Synapse Analytics versus Microsoft Fabric: A Side by Side Comparison
In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.

Data validation in Python: a look into Pandera and Great Expectations
Implement Python data validation with Pandera & Great Expectations in this comparison of their features and use cases.

How to setup Python, PyEnv & Poetry on Windows
Explore using Python virtual environments & Poetry on Windows for smoother workflows, with a script & guide to enhance your dependency management experience.
Security and Compliance
View all (27)
Exploring OpenChain: From License Compliance to Security Assurance
Open-source software has become an essential part of many organisation's software supply chain, however, this poses challenges with license compliance and security assurance.

The OpenChain specification explained
When implementing OpenChain, understanding the specification will help guide your organisation to having processes in place to review and manage open-source software

What are the risks with open-source software?
The key risks associated with open-source software, from whether you use it minimally, to using it throughout all your systems.
Startups
View all (15)
How to Monetize APIs with Azure API Management
Monetizing your APIs is a complex challenge; creating your business, revenue, & pricing model, then translating into a technical implementation requires significant investment. Working with the Azure API Management team we provide guidance, videos and code to help accelerate your journey.

10 ways working with Microsoft helped endjin grow since 2010
Microsoft recently shot a video interviewing endjin co-founder, Howard van Rooijen, and Director of Engineering, James Broome, about how Microsoft has helped endjin grow over the past decade. This posts the top 10 ways in which Microsoft helped - from providing access to valuable software and services, to opening up sales channels, to helping to navigate the minefield of UK Financial Services regulations around cloud adoption.

What makes a successful FinTech start-up?
In this post we discuss the characteristics of a great FinTech startup, and the importance of the API Economy to innovation in Financial Services.
Strategy
View all (64)
Agile teams often get their planning and priorities wrong
Do you struggle to manage conflicting demands from multiple stakeholders, even when they have agreed about the relative priority of their requests? Apply a budgeting methodology to agile projects for better planning and communication with the team and wider stakeholders.

Ask the right questions to get your data insights projects back on track
Learn about the thinking behind endjin's Power BI Maturity assessment by applying Wardley Doctrine, and asking more questions.

SQLbits 2023 - The Best Bits
This is a summary of the sessions I attended at SQLbits 2023 in Newport Wales, which is Europe's largest expert led data conference.
UX
View all (19)
How to develop an accessible colour palette for Power BI
If the audience for your Power BI report is larger than 12 people, then statistically 1 of your audience suffers from Colour Vision Deficiency. In this post we will talk through how we experimented with colour to develop an accessible colour palette and why colour and contrast is important when it comes to data visualisation.

How to enable data teams with the design assets required for impactful data storytelling in Power BI
In this post we will talk through how to expand on a data team's creative skillset, without access to specialist photo editing software such as Photoshop or Illustrator.

How to dynamically switch between measures in Power BI visuals with Field Parameters
In this blog post explore using Field Parameters to dynamically switch between measures in visuals, enhancing report analysis capabilities.
Visualisation
View all (12)
How to develop an accessible colour palette for Power BI
If the audience for your Power BI report is larger than 12 people, then statistically 1 of your audience suffers from Colour Vision Deficiency. In this post we will talk through how we experimented with colour to develop an accessible colour palette and why colour and contrast is important when it comes to data visualisation.

How to enable data teams with the design assets required for impactful data storytelling in Power BI
In this post we will talk through how to expand on a data team's creative skillset, without access to specialist photo editing software such as Photoshop or Illustrator.

How to dynamically switch between measures in Power BI visuals with Field Parameters
In this blog post explore using Field Parameters to dynamically switch between measures in visuals, enhancing report analysis capabilities.