Skip to content
  1. We help small teams achieve big things.
  2. Our Technical Fellow just became a Microsoft MVP!
  3. We run the Azure Weekly newsletter.
  4. We just won 2 awards.
  5. We just published Programming C# 8.0 book.
  6. We run the Power BI Weekly newsletter.
  7. Our NDC London 2020 is now available online!
  8. We are school STEM ambassadors.
Quick tip – Updating the sort order of a column in Power BI (avoiding circular references)

Quick tip – Updating the sort order of a column in Power BI (avoiding circular references)

Carmel Eve

Here's a quick tip on how to alter the sort order of a column in Power BI!
Talking about Azure Synapse on Microsoft Mechanics!

Talking about Azure Synapse on Microsoft Mechanics!

Jess Panni

I was recently invited on to Microsoft Mechanics to talk about the new on-demand SQL Serverless offering within Azure Synapse. If you have been following along with my previous blog posts you will know that we've been hard at work applying Azure Synapse against real customer workloads. In the video I take you through the service by solving a real-world IoT problem for one of our telco customers.
Benchmarking Azure Synapse Analytics - SQL Serverless, using .NET Interactive

Benchmarking Azure Synapse Analytics - SQL Serverless, using .NET Interactive

Jess Panni

There is a new service in town that promises to transform the way you query the contents of your data lake. Azure Synapse Analytics comes with a new offering called SQL Serverless allowing you to query your data on-demand with no need for pre-provisioned resources.When we heard about the new service we were keen to get involved, so for the last 10 months we've been working with the SQL Serverless product group to provide feedback on the service and to help ensure it meets our customers needs. During this time we've put it through it's paces by implementing a range of real-world use cases. We were particularly interested to see how it stacked up as a replacement for Data Lake Analytics, where to date there has been no clear and easy migration path.
Does Azure Synapse Analytics spell the end for Azure Databricks?

Does Azure Synapse Analytics spell the end for Azure Databricks?

Jess Panni

Have you or are you about to invest in Azure Databricks? If so, the new Spark offering in Azure Synapse Analytics is likely to have grabbed your attention and rightly so. Why is Microsoft putting yet another Spark offering on the table and what does it mean for you?
5 Reasons why Azure Synapse Analytics should be on your roadmap

5 Reasons why Azure Synapse Analytics should be on your roadmap

Jess Panni

For years we have been building modern cloud data solutions on Azure and helping our customers transform their use of data to drive outcomes. Here are 5 reasons why Azure Synapse Analytics might just be the service that we have been crying out for.
How can I improve my data model in Power BI?

How can I improve my data model in Power BI?

Carmel Eve

Configuring model properties in Power BI allows you to create a model which is far more discoverable and is able to better support the visualisations you need. There are several different model properties which can be configured, some of these focus on discoverability whilst others allow you to alter the ways in which data is sorted/displayed/summarised in the reports.
Learning DAX and Power BI – CALCULATE

Learning DAX and Power BI – CALCULATE

Carmel Eve

This is the final blog in a series about DAX and Power BI. This post focuses on the CALCULATE function, which is a unique function in DAX. The CALCULATE function has the ability to alter filter contexts, and therefore can be used to enable extremely powerful and complex processing. This post covers some of the most common scenarios for using CALCULATE, and some of the gotchas in the way in which these different features interact!
Learning DAX and Power BI – Related Tables and Relationships

Learning DAX and Power BI – Related Tables and Relationships

Carmel Eve

This is the sixth blog in a series about DAX and Power BI. This post focuses on relationships and related tables. These relationships allow us to build up intricate and powerful models using a combination of sources and tables. The use of relationships in DAX powers many of the features around slicing and page filtering of reports.
Learning DAX and Power BI – Table Functions

Learning DAX and Power BI – Table Functions

Carmel Eve

This is the fifth blog in a series on DAX and Power BI. This post focuses on table functions. In DAX, table functions return a table which can then be used for future processing. This can be useful if, for example, you want to perform an operation over a filtered dataset. Table functions, like most functions in DAX, operate under the filter context in which they are applied.
Building a proximity detection pipeline

Building a proximity detection pipeline

Carmel Eve

At endjin, our approach focuses on using scientific experimental method to support the creation of fully proved and tested decision making, and the use of scientific research to support our work. This post runs through how we applied that process to creation a pipeline to detect vessel proximity.This is an example which is based around the project we recently worked on with OceanMind. In this project we helped them to build a #serverless architecture which could detect vessel proximity in close to real time. The vessel proximity events we detected were then fed into machine learning algorithms in order to detect illegal fishing!Carmel also runs through some of the actual calculations we used to detect proximity, how we used #data projections to efficiently process large quantitities of incoming data, and the use of #durablefunctions to orchestrate the processing.
Learning DAX and Power BI - Aggregators

Learning DAX and Power BI - Aggregators

Carmel Eve

This is the fourth blog in a series about DAX and Power BI. We have so far covered filter and row contexts, and the difference between calculated columns and measures. This post focuses on aggregators. We cover the limitations of the classic aggregators, and demontrate the power of the iterative versions. We also highlight some of the less intuitive features around how these functions interact with both filter and row contexts.
Learning DAX and Power BI – Calculated Columns and Measures

Learning DAX and Power BI – Calculated Columns and Measures

Carmel Eve

This is the third blog in a series about learning DAX and Power BI. The first two blogs focused on filter and row contexts. We are now moving on to talk about calculated columns and measures. These are the main features used to support the display of complex visuals. They allow you to combine columns, aggregate values, reformat data, and much more. The difference between these features can get a bit confusing so we've attempted to make that clearer using some concrete examples!
Optimising C# for a serverless environment

Optimising C# for a serverless environment

Carmel Eve

In our recent project with OceanMind we used #AzureFunctions to process marine vessel telemetry from around the world. This involved processing huge quantities of data in close to real time. We optimised our processing for a #serverless environment, the outcome of which being that the compute would cost less than £10 / month!This post summarises some of the techniques we used, including some concrete examples of optimisations we made.#bigdata #dataprocessing #dataanalysis #bigcompute
Learning DAX and Power BI – Row Contexts

Learning DAX and Power BI – Row Contexts

Carmel Eve

Here is the second blog in a series around learning DAX and Power BI. This post focuses on row contexts, which are used when iterating over the rows of a table when, for example, evaluating a calculated column. Row contexts along with filter contexts underpin the basis of the DAX language. Once you understand this underlying theory it is purely a case of learning the syntax for the different operations which are built on top of it.
Learning DAX and Power BI - Filter Contexts

Learning DAX and Power BI - Filter Contexts

Carmel Eve

Here is the first in a series of blog posts around understanding DAX and Power BI. This post focuses on filter contexts. which are a central concept which is vital for being able to write effective and powerful DAX!In this series Carmel walks through the main ideas and syntax surrounding the DAX language, and provides examples of using these over a dataset. DAX is an extremely powerful language. Using these techniques it is possible to build up complex reports which provide the insight you really need!
Exposing legacy batch processing code online using Azure Durable Functions, API Management and Kubernetes

Exposing legacy batch processing code online using Azure Durable Functions, API Management and Kubernetes

Jonathan George

In this post we show how a combination of Kubernetes, Azure Durable Functions and Azure API Management can be used to make legacy batch processing code available as a RESTful API. This is a great example of how serverless technologies can be used to expose legacy software to the public internet in a controlled way, allowing you to reap some of the benefits of a cloud first approach without fully rewriting and migrating existing software.
NDC London 2020 - My highlights

NDC London 2020 - My highlights

Ed Freeman

Ed attended NDC London 2020, along with many of his endjin colleagues. In this post he summarises and reflects upon his favourite sessions of the conference including; "OWASP Top Ten proactive controls" by Jim Manico, "There's an Impostor in this room!" by Angharad Edwards, "How to code music?" by Laura Silvanavičiūtė, "ML and the IoT: Living on the Edge" by Brandon Satrom, "Common API Security Pitfalls" by Philippe De Ryck, and "Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month" by Jess Panni & Carmel Eve.
NDC London - A dive into responsible and inclusive technology

NDC London - A dive into responsible and inclusive technology

Carmel Eve

NDC London day 1 was mainly focused around the responsibility we all face when developing new technology. As developers we cannot absolve ourselves of the consequences of not considering diversity and inclusivity when designing our solutions.
Building a secure data solution using Azure Data Lake Store (Gen2)

Building a secure data solution using Azure Data Lake Store (Gen2)

Carmel Eve

In this blog from the Azure Advent Calendar 2019 we discuss building a secure data solution using Azure Data Lake. Data Lake has many features which enable fine grained security and data separation. It is also built on Azure Storage which enables us to take advantage of all of those features and means that ADLS is still a cost effective storage option!This post runs through some of the great features of ADLS and runs through an example of how we build our solutions using this technology!

Speaking at NDC London: Combatting illegal fishing with Machine Learning and Azure

Carmel Eve

In January 2020, Carmel is speaking about creating high performance geospatial algorithms in C# which can detect suspicious vessel activity, which is used to help alert law enforcement to illegal fishing. The input data is fed from Azure Data Lake Storage Gen 2, and converted into data projections optimised for high-performance computation. This code is then hosted in Azure Functions for cheap, consumption based processing.
How Azure DevTestLabs is helping me climb Everest

How Azure DevTestLabs is helping me climb Everest

Carmel Eve

Remote working allows us to work from anywhere we want. This brings a huge amount of flexibility in freedom, however we do need the help of a working laptop! When Carmel's laptop gave in just before a trip, she used Azure DevTestLabs to allow her to continue to work using a 10 year old Mac that probably couldn't wouldn't have been up to the task alone...
Demystifying machine learning using neural networks

Demystifying machine learning using neural networks

Carmel Eve

Machine learning often seems like a black box. This post walks through what's actually happening under the covers, in an attempt to de-mystify the process!Neural networks are built up of neurons. In a shallow neural network we have an input layer, a "hidden" layer of neurons, and an output layer. For deep learning, there is simply more hidden layers which allows for combining neuron's inputs and outputs to build up a more detailed picture.If you have an interest in Machine Learning and what is really happening, definitely give this a read (WARNING: Some algebra ahead...)!
Secure function-to-function authentication in Azure without the need for credentials

Secure function-to-function authentication in Azure without the need for credentials

Carmel Eve

Building a secure solution on Azure can be a daunting task. Using Azure Functions and Managed Identities, we have built up a pattern for giving services access to one another, woithout the need to store credentials. These managed identities can be given access to necessary resources. For example, they can be granted roles and added to access control lists in ADLS Gen2 accounts, or the ability to access keys in key vault. This means that data can be securely accessed without needing to store connection strings or app passwords.
Design patterns in C# - The Builder Pattern

Design patterns in C# - The Builder Pattern

Carmel Eve

This is the second blog in a series around design patterns. This post focuses on the builder pattern. The builder pattern is used when there is complex set up involved in creating an object. Like the other creational patterns, it also separates out the construction of an object from the object's use.
Using Databricks Notebooks to run an ETL process

Using Databricks Notebooks to run an ETL process

Carmel Eve

Here at endjin we've done a lot of work around data analysis and ETL. As part of this we have done some work with Databricks Notebooks on Microsoft Azure. Notebooks can be used for complex and powerful data analysis using Spark. Spark is a "unified analytics engine for big data and machine learning". It allows you to run data analysis workloads, and can be accessed via many APIs. This means that you can build up data processes and models using a language you feel comfortable with. They can also be run as an activity in a ADF pipeline, and combined with Mapping Data Flows to build up a complex ETL process which can be run via ADF.
Exploring Azure Data Factory - Mapping Data Flows

Exploring Azure Data Factory - Mapping Data Flows

Carmel Eve

Mapping Data Flows are a relatively new feature of ADF. They allow you to visually build up complex data transformation sequences. This can aid in the streamlining of data manipulation and ETL processes, without the need to write any code! This post gives a brief introduction to the technology, and what this could enable!
A code review with NDepend Part 2: The initial review

A code review with NDepend Part 2: The initial review

Carmel Eve

At endjin we have a high quality bar when it comes to our code. As part of this we carry out regular code reviews. One of the tools we have used for these code reviews is NDepend. This is the second in a blog series written as we carried out that process. This post focuses on the insight you can quickly gain just by glancing at the NDepend UI.
A beginner's guide to agile estimation and planning

A beginner's guide to agile estimation and planning

Carmel Eve

In this post Carmel runs through some of the main principles behind agile estimation and planning. At endjin we use a lot of these techniques in our projects and this is a great post which highlights the reasons behind some of what we do. The key motivation behind good estimation is to be useful for project planning. There is a huge amount of inherent uncertainty surrounding estimates, especially early in the project. So, we shift our aim from 100% precise, or "true", estimates, and towards providing estimates which are useful and accurate. Carmel also runs through the steps in an agile delivery and release process. Definitely worth the read if you have an interest in agile and/or project management!

11 cheers for binary (And 3 for hexadecimal)!

Carmel Eve

Sometimes it's good to go back to the basics... This is a quick post that runs through binary and hexadecimal numbers, and how those relate to our every day computing!
Rx operators deep dive Part 4: A window into scheduling in Rx

Rx operators deep dive Part 4: A window into scheduling in Rx

Carmel Eve

This is the fourth blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the WINDOW operator.
Rx operators deep dive Part 3: Re-grouping our thoughts

Rx operators deep dive Part 3: Re-grouping our thoughts

Carmel Eve

This is the third blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the GROUP operator.

A conversation about .NET, The Cloud, Data & AI, teaching software engineers and joining endjin with Ian Griffiths

Ian Griffiths

When he joined endjin, Technical Fellow Ian sat down with founder Howard for a Q&A session. This was originally published on LinkedIn in 5 parts, but is republished here, in full. Ian talks about his path into computing, some highlights of his career, the evolution of the .NET ecosystem, AI, and the software engineering life.
Rx operators deep dive part 2: Slowly aggregating knowledge

Rx operators deep dive part 2: Slowly aggregating knowledge

Carmel Eve

This is the second blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the AGGREGATE operator.
Rx operators deep dive part 1: Where observables meet LINQ

Rx operators deep dive part 1: Where observables meet LINQ

Carmel Eve

This is the first blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the WHERE operator.
Overflowing with dataflow part 2: TPL Dataflow

Overflowing with dataflow part 2: TPL Dataflow

Carmel Eve

This is the second blog in a series about data flow. This post delves into TPL dataflow.The task parallel library is a .NET library which aims to make parallel processing and concurrency simpler to work with. The TPL dataflow library is specifically aimed at making parallel data processing more understandable via a pipeline-based model.
Thoughts about .NET, The Cloud, AI, ML, and teaching software engineers

Thoughts about .NET, The Cloud, AI, ML, and teaching software engineers

Howard van Rooijen

A tentative step into the worlds of asymmetric encryption and Blockchain

A tentative step into the worlds of asymmetric encryption and Blockchain

Carmel Eve

Here is a quick dive into encryption and blockchain. This post goes into the ideas behind hashing, and how these translate into encrypted messaging techniques. It also delves blockchain and how signing and versioning allow for consistent and immutable transactions. Definitely worth a read if you're interested in these concepts!
OneNote - helping me to find my feet in research

OneNote - helping me to find my feet in research

Carmel Eve

Everyone learns differently. In this post Carmel describes how OneNote can be used to aid and enhance research. As an avid note taker and blogger, she highlights how the ability to Ctrl-F into written text in OneNote has greatly improved her productivity!
Simon Sinek's "Start With Why" is the prequel to Satya Nadella's "Hit Refresh"

Simon Sinek's "Start With Why" is the prequel to Satya Nadella's "Hit Refresh"

Howard van Rooijen

Kickstart your API proposition with the API Maturity Matrix

Kickstart your API proposition with the API Maturity Matrix

Howard van Rooijen

Embracing Disruption - Financial Services and the Microsoft Cloud

Embracing Disruption - Financial Services and the Microsoft Cloud

Howard van Rooijen

We have produced an insightful booklet called "Embracing Disruption - Financial Services and the Microsoft Cloud" which examines the challenges and opportunities for the Financial Service Industry in the UK, through the lens of Microsoft Azure, Security, Privacy & Data Sovereignty, Data Ingestion, Transformation & Enrichment, Big Compute, Big Data, Insights & Visualisation, Infrastructure, Ops & Support, and the API Economy.
What makes a successful FinTech start-up?

What makes a successful FinTech start-up?

Matthew Adams

In this post we discuss the characteristics of a great FinTech startup, and the importance of the API Economy to innovation in Financial Services.
The 100 Year Start-up: Embracing Disruption in Financial Services

The 100 Year Start-up: Embracing Disruption in Financial Services

Howard van Rooijen

Hymans Robertson was set up in Glasgow in 1921 and is one of the longest established independent firms of consultants and actuaries in the UK. Hymans Robertson soon realised that the computational requirements of their models exceeded the capacity of their on-premise datacentres and that the most cost effective solution would be to use the cloud to perform their Big Compute. But before they could harness the cloud to help them solve their Big Data problems, the business needed to understand the ramifications of moving to the cloud; everything from regulatory, risk and compliance concerns,  to how their internal Ops team would need to evolve and adapt, and how to deal with moving data from on-prem into the cloud.
Why is blockchain revolutionising Financial Services?

Why is blockchain revolutionising Financial Services?

Matthew Adams

There is a lot of hype about the blockchain - usually wrapped up with talk about Bitcoin and crypto-currencies. In this article, we look at its impact on trust, and auditability in financial services, and why it may (or may not) be appropriate for your solutions.
Regulatory Compliance and Cloud Adoption

Regulatory Compliance and Cloud Adoption

Matthew Adams

In this post we review the FCA's guidelines for the adoption of cloud services by FinTech businesses, and help you to understand their impact across the value chain.
FinTech Week and the Microsoft Cloud

FinTech Week and the Microsoft Cloud

Matthew Adams