Skip to content
Azure Synapse Analytics: How serverless is replacing the data warehouse

Azure Synapse Analytics: How serverless is replacing the data warehouse

James Broome

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?
Benchmarking Azure Synapse Analytics - SQL Serverless, using Polyglot Notebooks

Benchmarking Azure Synapse Analytics - SQL Serverless, using Polyglot Notebooks

James Broome

New Azure Synapse Analytics service offers SQL Serverless for on-demand data lake queries. We tested its potential as a Data Lake Analytics replacement.
Does Azure Synapse Analytics spell the end for Azure Databricks?

Does Azure Synapse Analytics spell the end for Azure Databricks?

James Broome

Explore why Microsoft's new Spark offering in Azure Synapse Analytics is a game-changer for Azure Databricks investors.
Recording of Azure Oxford talk on combatting illegal fishing with Azure (for less than £10/month)

Recording of Azure Oxford talk on combatting illegal fishing with Azure (for less than £10/month)

Carmel Eve

Jess and Carmel recently gave a talk at Azure Oxford on Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month. The recording of that talk is now available for viewing!The talk focuses on the recent work we completed with OceanMind. They run through how to construct a cloud-first architecture based on serverless and data analytics technologies and explore the important principles and challenges in designing this kind of solution. Finally, we see how the architecture we designed through this process not only provides all the benefits of the cloud (reliability, scalability, security), but because of the pay-as-you-go compute model, has a compute cost that we could barely believe!
Building a proximity detection pipeline

Building a proximity detection pipeline

Carmel Eve

Endjin's blog post details their project with OceanMind, using a serverless architecture and machine learning to detect illegal fishing.
Optimising C# for a serverless environment

Optimising C# for a serverless environment

Carmel Eve

Tour our OceanMind project using Azure Functions for real-time marine telemetry processing. Learn optimization techniques for serverless environments.
Building a secure data solution using Azure Data Lake Store (Gen2)

Building a secure data solution using Azure Data Lake Store (Gen2)

Carmel Eve

In this blog we discuss building a secure data solution using Azure Data Lake. Data Lake has many features which enable fine grained security and data separation. It is also built on Azure Storage which enables us to take advantage of all of those features and means that ADLS is still a cost effective storage option!This post runs through some of the great features of ADLS and runs through an example of how we build our solutions using this technology!

Speaking at NDC London: Combatting illegal fishing with Machine Learning and Azure

Carmel Eve

In January 2020, Carmel is speaking about creating high performance geospatial algorithms in C# which can detect suspicious vessel activity, which is used to help alert law enforcement to illegal fishing. The input data is fed from Azure Data Lake Storage Gen 2, and converted into data projections optimised for high-performance computation. This code is then hosted in Azure Functions for cheap, consumption based processing.
C#, Span and async

C#, Span and async

Ian Griffiths

Explore how ref struct types like Span<T> in C# enhance performance but pose async method challenges, and learn mitigation techniques.
Increasing performance via low memory allocation in C#

Increasing performance via low memory allocation in C#

Carmel Eve

Explore techniques for high-performance, low-allocation code in Azure Functions using C#, including data streaming, list preallocation, and Span&lt;T&gt;.
Running Azure Functions in Docker on a Raspberry Pi 4

Running Azure Functions in Docker on a Raspberry Pi 4

Jonathan George

Running an Azure Function in a Docker container on Raspberry Pi 4 with this step-by-step guide and ready-to-use code for your projects.
Import and export notebooks in Databricks

Import and export notebooks in Databricks

Ed Freeman

Learn to import/export notebooks in Databricks workspaces manually or programmatically, and transfer content between workspaces efficiently.
Demystifying machine learning using neural networks

Demystifying machine learning using neural networks

Carmel Eve

Machine learning often seems like a black box. This post walks through what's actually happening under the covers, in an attempt to de-mystify the process!Neural networks are built up of neurons. In a shallow neural network we have an input layer, a "hidden" layer of neurons, and an output layer. For deep learning, there is simply more hidden layers which allows for combining neuron's inputs and outputs to build up a more detailed picture.If you have an interest in Machine Learning and what is really happening, definitely give this a read (WARNING: Some algebra ahead...)!
Using Databricks Notebooks to run an ETL process

Using Databricks Notebooks to run an ETL process

Carmel Eve

Explore data analysis & ETL with Databricks Notebooks on Azure. Utilize Spark's unified analytics engine for big data & ML, and integrate with ADF pipelines.
Exploring Azure Data Factory - Mapping Data Flows

Exploring Azure Data Factory - Mapping Data Flows

Carmel Eve

Mapping Data Flows are a relatively new feature of ADF. They allow you to visually build up complex data transformation sequences. This can aid in the streamlining of data manipulation and ETL processes, without the need to write any code! This post gives a brief introduction to the technology, and what this could enable!
Avoiding deployment locking errors by running Web and Functions Apps from packages

Avoiding deployment locking errors by running Web and Functions Apps from packages

Carmel Eve

Fix DLL locking errors in Azure Function deployment by using "deploy from package" option. This runs the function from a package file, resolving issues.
A conversation about .NET, The Cloud, Data & AI, teaching software engineers and joining endjin with Ian Griffiths

A conversation about .NET, The Cloud, Data & AI, teaching software engineers and joining endjin with Ian Griffiths

Ian Griffiths

When he joined endjin, Technical Fellow Ian sat down with founder Howard for a Q&A session. This was originally published on LinkedIn in 5 parts, but is republished here, in full. Ian talks about his path into computing, some highlights of his career, the evolution of the .NET ecosystem, AI, and the software engineering life.
Using Python inside SQL Server

Using Python inside SQL Server

Ed Freeman

Learn to use SQL Server's Python integration for efficient data handling. Eliminate clunky transfers and easily operationalize Python models/scripts.
Snap Back to Reality – Month 2 & 3 of my Apprenticeship

Snap Back to Reality – Month 2 & 3 of my Apprenticeship

Ed Freeman

Learn what types of things an apprentice gets up to at endjin a few months after joining. You could be learning about Neural Networks: algorithms which mimic the way biological systems process information. You could be attending Microsoft's Future Decoded conference, learning about Bots, CosmosDB, IoT and much more. Hopefully, you wouldn't be in hospital after a ruptured appendix!
How to plan your cloud transformation journey

How to plan your cloud transformation journey

Howard van Rooijen

We've been helping customers adopt Microsoft Azure since 2010, we have produced a lot of thought leadership to help people think about the steps required, the risk involved and how to plan a successful adoption.
AWS vs Azure vs Google Cloud Platform - Mobile Services

AWS vs Azure vs Google Cloud Platform - Mobile Services

Jess Panni

AWS vs Azure vs Google Cloud Platform - Database

AWS vs Azure vs Google Cloud Platform - Database

Jess Panni

AWS vs Azure vs Google Cloud Platform - Compute

AWS vs Azure vs Google Cloud Platform - Compute

Jess Panni

Embracing Disruption - Financial Services and the Microsoft Cloud

Embracing Disruption - Financial Services and the Microsoft Cloud

Howard van Rooijen

We have produced an insightful booklet called "Embracing Disruption - Financial Services and the Microsoft Cloud" which examines the challenges and opportunities for the Financial Service Industry in the UK, through the lens of Microsoft Azure, Security, Privacy & Data Sovereignty, Data Ingestion, Transformation & Enrichment, Big Compute, Big Data, Insights & Visualisation, Infrastructure, Ops & Support, and the API Economy.
Azure Batch - Time is Money in Big Compute

Azure Batch - Time is Money in Big Compute

James Broome

Consumption based pricing is a one of the USPs of Cloud PaaS services, but the default settings aren't necessarily optimised for cost. Significant savings can be made from understanding your workload.
Azure Machine Learning–experimenting with training data proportions using the SMOTE module

Azure Machine Learning–experimenting with training data proportions using the SMOTE module

Alice Waddicor

Spinning up 16,000 A1 Virtual Machines on Azure Batch

Spinning up 16,000 A1 Virtual Machines on Azure Batch

Howard van Rooijen

We recently completed a technical proof of concept to see if the new Azure Batch service could scale to meet the demands of a Big Compute workload.