Editions

The first in a series of posts talking about how to best work with configuration in Azure Functions

A rundown of how to trigger an Azure Synapse pipeline run from C# code.

Get started with Mattermost on Azure using the Mattermost on AKS quickstart from the Azure CNAB Quickstart Library.

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Did you know that there's a null-safe operator in Azure Data Factory's expression syntax? No? Well, now you do. Here's how you can safely reference an activity output that might not always exist.

Modern analytics requires a multi-faceted approach, which can cause integration headaches. Azure Synapse's Swiss army knife approach can remove a lot of friction.

At endjin we solve complex problems which cover a huge range of cloud services, data science techniques and industries. The work we do is massively varied and in this there is a huge opportunity. Hear about Carmel's experience in completing her apprenticeship here at endjin, and why you should consider our apprenticeship scheme!

We all get nervous in the run up to a public speaking event. However, there are things we can do which help alleviate some of the pressure. Here are 5 quick tips around preparing for a talk!

There are many different paths into the tech industry, Carmel has been speaking at some local schools about joining the industry from a scientific background. In this post she discusses the crucial tools which science gives you which can help you succeed in tech!

This blog is part of a series around design patterns. This post focuses on the proxy pattern. The proxy pattern is used to restrict and control access to underlying objects. It is useful in communicating with remote servers, or in adding extra functionality on object access.

This blog is part of a series around design patterns. This post focuses on the facade pattern. The facade pattern provides an interface into a complex system, in order to simplify access to the functionality.

This blog is part of a series around design patterns. This post focuses on the adapter pattern. The adapter pattern is used to enable you to use the functionality defined by a class which is not compatible with the interfaces/architecture of your solution.

The first in a series of posts talking about how to best work with configuration in Azure Functions

Did you know that there's a null-safe operator in Azure Data Factory's expression syntax? No? Well, now you do. Here's how you can safely reference an activity output that might not always exist.

A rundown of how to trigger an Azure Synapse pipeline run from C# code.

Did you know that there's a null-safe operator in Azure Data Factory's expression syntax? No? Well, now you do. Here's how you can safely reference an activity output that might not always exist.

A rundown of how to trigger an Azure Synapse pipeline run from C# code.

Modern analytics requires a multi-faceted approach, which can cause integration headaches. Azure Synapse's Swiss army knife approach can remove a lot of friction.

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?

There is a new service in town that promises to transform the way you query the contents of your data lake. Azure Synapse Analytics comes with a new offering called SQL Serverless allowing you to query your data on-demand with no need for pre-provisioned resources.When we heard about the new service we were keen to get involved, so for the last 10 months we've been working with the SQL Serverless product group to provide feedback on the service and to help ensure it meets our customers needs. During this time we've put it through it's paces by implementing a range of real-world use cases. We were particularly interested to see how it stacked up as a replacement for Data Lake Analytics, where to date there has been no clear and easy migration path.

Have you or are you about to invest in Azure Databricks? If so, the new Spark offering in Azure Synapse Analytics is likely to have grabbed your attention and rightly so. Why is Microsoft putting yet another Spark offering on the table and what does it mean for you?

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Did you know that there's a null-safe operator in Azure Data Factory's expression syntax? No? Well, now you do. Here's how you can safely reference an activity output that might not always exist.

Summary of key themes from the Doing Data Together conference hosted virtually by The Scotsman newspaper and Edinburgh University in November 2020. The conference agenda was pivoted to focus on the use of data to help tackle the Covid-19 pandemic. It provided a fascinating insight into the lessons learned.

Summary of key themes from the Doing Data Together conference hosted virtually by The Scotsman newspaper and Edinburgh University in November 2020. The conference agenda was pivoted to focus on the use of data to help tackle the Covid-19 pandemic. It provided a fascinating insight into the lessons learned.

CodeOps enables you to layer standards, automation, and governance to your source control repositories, decreasing manual effort and increasing productivity.

Building a system that solves even just 10% of a problem is a worthwhile endeavour, not just because of the value the 10% delivers but because its mere existence will lower the barrier-to-entry for your colleagues to collaboratively iterate on it - not so much 'Build it and they will come' as 'Build a road and they will come'

Sometimes the logic for the bit of work you're doing in PowerShell won't be packaged into a handy PowerShell module with cmdlets to perform the operations you need to perform. But what if there is a .NET package listed on NuGet that does contain the assemblies you need for your work? How can we import that into PowerShell and make use of its classes and methods? This blog explains how to do just that, avoiding current pitfalls in PowerShell's implementation of the NuGet package provider.

Automating the updating of Power BI data source credentials is a common requirement in the application lifecycle management of a Power BI solution. However, on-premises data sources throw an extra spanner in the works: you must encrypt your credentials using the RSA-OAEP encryption algorithm before sending off the request to update the data source. There is currently no PowerShell module that encapsulates this logic into a set of handy cmdlets, so you must either implement the algorithm's logic yourself, or you can use the helper classes in the existing .NET SDK to do all the heavy lifting for you. This blog shows how to do the latter.

CodeOps enables you to layer standards, automation, and governance to your source control repositories, decreasing manual effort and increasing productivity.

Data insights are useless, and even dangerous (as we've seen recently at Public Health England) if they can't be trusted. So, the need to validate business rules and security boundaries within a data solution is critical. This post argues that if you're doing anything serious with data, then you should be taking this seriously.

This post describes the reasons for choosing specific & immutable dependency versions in your software.

GitOps (sometimes known as 'operations by pull request') is most often discussed as being an approach for managing your Kubernetes platform, but in this post I will make the case for how it can benefit a much wider audience.

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?

I was recently invited on to Microsoft Mechanics to talk about the new on-demand SQL Serverless offering within Azure Synapse. If you have been following along with my previous blog posts you will know that we've been hard at work applying Azure Synapse against real customer workloads. In the video I take you through the service by solving a real-world IoT problem for one of our telco customers.

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

Tracking down unexpected allocations in a high-performance .NET parsing library.

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Summary of key themes from the Doing Data Together conference hosted virtually by The Scotsman newspaper and Edinburgh University in November 2020. The conference agenda was pivoted to focus on the use of data to help tackle the Covid-19 pandemic. It provided a fascinating insight into the lessons learned.

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

Get started with Mattermost on Azure using the Mattermost on AKS quickstart from the Azure CNAB Quickstart Library.

Many API frameworks implement an async HTTP API pattern to address the problem of coordinating the state of long-running operations with external clients. This post looks at how we can use Axios interceptors to simplify and centralise the UI logic required to work with this API pattern.

Tracking down unexpected allocations in a high-performance .NET parsing library.

Automating the updating of Power BI data source credentials is a common requirement in the application lifecycle management of a Power BI solution. However, on-premises data sources throw an extra spanner in the works: you must encrypt your credentials using the RSA-OAEP encryption algorithm before sending off the request to update the data source. There is currently no PowerShell module that encapsulates this logic into a set of handy cmdlets, so you must either implement the algorithm's logic yourself, or you can use the helper classes in the existing .NET SDK to do all the heavy lifting for you. This blog shows how to do the latter.

In this post I talk about custom Power BI themes and share some handy tips for styling Power BI reports.

Data insights are useless, and even dangerous (as we've seen recently at Public Health England) if they can't be trusted. So, the need to validate business rules and security boundaries within a data solution is critical. This post argues that if you're doing anything serious with data, then you should be taking this seriously.

Data Engineers/Developers want to get access to Azure Synapse Analytics as quickly as possible to start designing and creating their data solutions. Being denied access to Synapse Studio can be frustrating and slows matters down. This article will address the "You need permission to access workspace..." error, discuss what causes it, and describe how to fix it.

Azure Synapse Analytics developers need to be assigned a role within Synapse Studio in order to access the GUI. The Azure CLI provides one way of programmatically achieving this, which can be done by any Owner or Contributor of the Azure Synapse Analytics resource.

Despite the subsequent media reporting, the loss of 16,000 Covid-19 test results at Public Health England wasn't caused by Excel. This post argues that a lack of an appropriate risk and mitigation analysis left the process exposed to human error, which ultimately led to the loss of data and inaccurate reporting. It describes a simple process that could have been applied to prevent the error, and how it will help if you're worried about ensuring quality or reducing risk in your own business, technology or data programmes.

Microsoft recently shot a video interviewing endjin co-founder, Howard van Rooijen, and Director of Engineering, James Broome, about how Microsoft has helped endjin grow over the past decade. This posts the top 10 ways in which Microsoft helped - from providing access to valuable software and services, to opening up sales channels, to helping to navigate the minefield of UK Financial Services regulations around cloud adoption.

In this post we discuss the characteristics of a great FinTech startup, and the importance of the API Economy to innovation in Financial Services.

There is a lot of hype about the blockchain - usually wrapped up with talk about Bitcoin and crypto-currencies. In this article, we look at its impact on trust, and auditability in financial services, and why it may (or may not) be appropriate for your solutions.

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?

Summary of key themes from the Doing Data Together conference hosted virtually by The Scotsman newspaper and Edinburgh University in November 2020. The conference agenda was pivoted to focus on the use of data to help tackle the Covid-19 pandemic. It provided a fascinating insight into the lessons learned.

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

In this post I talk about custom Power BI themes and share some handy tips for styling Power BI reports.

Making your Power BI reports accessible is crucial, and one important aspect is legibility. If you're using background colours to represent certain categories, then you need any text that's used in conjunction with that background to be as clear as possible. This blog demonstrates how to dynamically select the best font colour to overlay on any chosen background colour.

A few years ago, I attended a Guardian 1-day Masterclass - Introduction to Data Visualisation. What I did not know then was that it would have a profound effect on how I view design. This blog is a recap of that day and reference points to consider when designing and creating graphics for data visualisation.

In this post I talk about custom Power BI themes and share some handy tips for styling Power BI reports.

Making your Power BI reports accessible is crucial, and one important aspect is legibility. If you're using background colours to represent certain categories, then you need any text that's used in conjunction with that background to be as clear as possible. This blog demonstrates how to dynamically select the best font colour to overlay on any chosen background colour.

A few years ago, I attended a Guardian 1-day Masterclass - Introduction to Data Visualisation. What I did not know then was that it would have a profound effect on how I view design. This blog is a recap of that day and reference points to consider when designing and creating graphics for data visualisation.