Skip to content
How to access choice labels from Azure Synapse Link for Dataverse with SQL

How to access choice labels from Azure Synapse Link for Dataverse with SQL

Jessica Hill

Learn how to access the choice labels from Azure Synapse Link for Dataverse using T-SQL through SQL Serverless and by using Spark SQL in a Synapse Notebook.
Styling and Enhancing Model Driven Apps in Power Apps

Styling and Enhancing Model Driven Apps in Power Apps

Paul Waller

Discover a concise guide on improving Model Driven Power Apps styles with step-by-step instructions for a polished user experience.
How to access choice labels from Azure Synapse Link for Dataverse with PySpark

How to access choice labels from Azure Synapse Link for Dataverse with PySpark

Jessica Hill

Learn how to access the choice labels from Azure Synapse Link for Dataverse using PySpark.
Notebooks in Azure Databricks

Notebooks in Azure Databricks

Jessica Hill

This blog post explores interactive notebooks in Azure Databricks. An Azure Databricks Notebook is a powerful data science tool that support exploratory data analysis, hypothesis testing, data cleaning and transformation, data visualisation, statistical modeling and machine learning.
Copilot - Are You Ready to Unleash the Power of AI in Self Service Analytics?

Copilot - Are You Ready to Unleash the Power of AI in Self Service Analytics?

James Broome

Explore AI-powered self-service reporting with tools like Copilot in Power BI and Microsoft Fabric, balancing benefits and pitfalls.
Microsoft Fabric: Announced

Microsoft Fabric: Announced

Ed Freeman

Microsoft Fabric unifies Power BI, Data Factory & Data Lake on Synapse infrastructure, reducing cost & time while enabling citizen data science.
What is OneLake?

What is OneLake?

Ed Freeman

Explore OneLake, Microsoft Fabric's core storage for data in Azure & other clouds. Discover its role in Fabric workloads, the OneDrive equivalent for data storage.
Intro to Microsoft Fabric

Intro to Microsoft Fabric

Ed Freeman

Microsoft Fabric unifies data & analytics, building on Azure Synapse Analytics for improved data-level interoperability. Explore its offerings & pros/cons.
Notebooks in Azure Synapse Analytics

Notebooks in Azure Synapse Analytics

Jessica Hill

Discover the use of Synapse Notebooks in Azure Synapse Analytics for data analysis, cleaning, visualization, and machine learning.
Version Control in Databricks

Version Control in Databricks

Elisenda Gascon

Explore how to implement source control in Databricks notebooks, promoting software engineering best practices.
Ask the right questions to get your data insights projects back on track

Ask the right questions to get your data insights projects back on track

Matthew Adams

Learn about the thinking behind endjin's Power BI Maturity assessment by applying Wardley Doctrine, and asking more questions.
SQLbits 2023 - The Best Bits

SQLbits 2023 - The Best Bits

Barry Smart

This is a summary of the sessions I attended at SQLbits 2023 in Newport Wales, which is Europe's largest expert led data conference.
Working with JSON in Pyspark

Working with JSON in Pyspark

Charlotte Gayton

This blog shows how to transform a document based format to a tabular format whilst managing all the nested properties
Data validation in Python: a look into Pandera and Great Expectations

Data validation in Python: a look into Pandera and Great Expectations

Liam Mooney

Implement Python data validation with Pandera & Great Expectations in this comparison of their features and use cases.
How to develop an accessible colour palette for Power BI

How to develop an accessible colour palette for Power BI

Paul Waller

Explore how we developed an accessible color palette for Power BI reports, considering color vision deficiency and data visualization.
Big Data LDN: highlights and how to survive your first data conference

Big Data LDN: highlights and how to survive your first data conference

Elisenda Gascon

Here are some of the highlights form this year's Big Data LDN conference.
Customizing Lake Databases in Azure Synapse Analytics

Customizing Lake Databases in Azure Synapse Analytics

Ed Freeman

Explore Custom Objects in Lake Databases for user-friendly column names, calculated columns, and pre-defined queries in Azure Synapse Analytics.
What is a Lake Database in Azure Synapse Analytics?

What is a Lake Database in Azure Synapse Analytics?

Ed Freeman

Explore Lake Databases in Azure Synapse Analytics: analyze Dataverse data, share Spark tables, and design models with Database Templates.
EVALUATEANDLOG in DAX

EVALUATEANDLOG in DAX

Jessica Hill

This blog post explores an exciting, hidden new feature in DAX - the EVALUATEANDLOG function. EVALUATEANDLOG brings debugging capability to Power BI.
Insight Discovery (part 6) – How to define business requirements for a successful cloud data & analytics project

Insight Discovery (part 6) – How to define business requirements for a successful cloud data & analytics project

James Broome

Many data projects fail to deliver the impact they should for a simple reason – they focus on the data. This series of posts explains a different way of thinking that will set up your data & analytics projects for success. Using an iterative, action-oriented, insight discovery process, it demonstrates tools and techniques that will help you to identify, define and prioritize requirements in your own projects so that they deliver maximum value. It also explores the synergy with modern cloud analytics platforms like Azure Synapse, explaining how the process and the architecture actively support each other for fast, impactful delivery.
Insight Discovery (part 5) – Deliver insights incrementally with data pipelines

Insight Discovery (part 5) – Deliver insights incrementally with data pipelines

James Broome

Discover a unique, action-oriented approach to data projects for maximum impact. Learn how to prioritize requirements and leverage cloud analytics platforms.
Insight Discovery (part 4) – Data projects should have a backlog

Insight Discovery (part 4) – Data projects should have a backlog

James Broome

This series focuses on maximizing data projects' impact via an iterative, insight discovery process, and synergy with cloud platforms like Azure Synapse.
Insight Discovery (part 3) – Defining Actionable Insights

Insight Discovery (part 3) – Defining Actionable Insights

James Broome

Discover a unique, action-oriented approach to data projects for maximum impact. Learn how to synergize with platforms like Azure Synapse for fast delivery.
How to enable data teams with the design assets required for impactful data storytelling in Power BI

How to enable data teams with the design assets required for impactful data storytelling in Power BI

Paul Waller

In this post we will talk through how to expand on a data team's creative skillset, without access to specialist photo editing software such as Photoshop or Illustrator.
5 tips to pass the PL-300 exam: Microsoft Power BI Data Analyst

5 tips to pass the PL-300 exam: Microsoft Power BI Data Analyst

Elisenda Gascon

I recently passed the PL-300 - Power BI Data Analyst exam. Here are some tips to prepare for it that I found useful!
Insight Discovery (part 2) – successful data projects start by forgetting about the data

Insight Discovery (part 2) – successful data projects start by forgetting about the data

James Broome

Many data projects fail to deliver the impact they should for a simple reason – they focus on the data. This series of posts explains a different way of thinking that will set up your data & analytics projects for success. Using an iterative, action-oriented, insight discovery process, it demonstrates tools and techniques that will help you to identify, define and prioritize requirements in your own projects so that they deliver maximum value. It also explores the synergy with modern cloud analytics platforms like Azure Synapse, explaining how the process and the architecture actively support each other for fast, impactful delivery.
Performance Optimisation Tools for Power BI

Performance Optimisation Tools for Power BI

Jessica Hill

Optimise Power BI report performance with analyzer tools. Discover essential techniques for efficient report development in this blog post.
Insight Discovery (part 1) – why do data projects often fail?

Insight Discovery (part 1) – why do data projects often fail?

James Broome

Discover a unique, action-oriented approach to data projects for maximum impact. Learn how to synergize with platforms like Azure Synapse for fast delivery.
Automating Excel in the Cloud with Office Scripts and Power Automate

Automating Excel in the Cloud with Office Scripts and Power Automate

Liam Mooney

Automate Excel tasks with Office Scripts & Power Automate. Get an overview and explore a practical example in this post.
How to apply behaviour driven development to data and analytics projects

How to apply behaviour driven development to data and analytics projects

Barry Smart

In this blog we demonstrate how the Gherkin specification can be adapted to enable BDD to be applied to data engineering use cases.
Sharing access to synchronized Shared Metadata Model objects in Azure Synapse Analytics

Sharing access to synchronized Shared Metadata Model objects in Azure Synapse Analytics

Ed Freeman

Learn how to grant non-admin users access to Spark synchronized objects with SQL Serverless in Synapse Analytics using the Shared Metadata Model.
What is the Shared Metadata Model in Azure Synapse Analytics, and why should I use it?

What is the Shared Metadata Model in Azure Synapse Analytics, and why should I use it?

Ed Freeman

Explore Azure Synapse's 'Shared Metadata Model' feature. Learn how it syncs Spark tables with SQL Serverless, its benefits, and tradeoffs.
Context Transition in DAX

Context Transition in DAX

Jessica Hill

This blog post explores how the CALCULATE function in DAX performs context transition.
Excel, data loss, IEEE754, and precision

Excel, data loss, IEEE754, and precision

Ian Griffiths

Explore the impact of Excel's numeric precision rules on identifiers and learn about infamous data loss incidents caused by misuse in this post.
CALCULATE in DAX

CALCULATE in DAX

Jessica Hill

This blog post explores one of the most powerful functions in DAX - CALCULATE.
RELATED and RELATEDTABLE in DAX

RELATED and RELATEDTABLE in DAX

Jessica Hill

This blog post explores two common functions in DAX - RELATED and RELATEDTABLE.
Extract insights from tag lists using Python Pandas and Power BI

Extract insights from tag lists using Python Pandas and Power BI

Barry Smart

Discover how to extract insights from spreadsheets and CSV files using Pandas and Power BI in this blog post.
Filtering unrelated tables in Power BI

Filtering unrelated tables in Power BI

Elisenda Gascon

Explore filtering fact tables with Star Schemas and learn a clean method for filtering dimension tables by another dimension.
How to dynamically switch between measures in Power BI visuals with Field Parameters

How to dynamically switch between measures in Power BI visuals with Field Parameters

Jessica Hill

In this blog post explore using Field Parameters to dynamically switch between measures in visuals, enhancing report analysis capabilities.
Table Functions in DAX: DISTINCT

Table Functions in DAX: DISTINCT

Jessica Hill

This blog post explores one of the most common table functions in DAX - DISTINCT.
Table Functions in DAX: FILTER and ALL

Table Functions in DAX: FILTER and ALL

Jessica Hill

This blog post explores two of the most common table functions in DAX - FILTER and ALL.
Measures vs Calculated Columns in DAX and Power BI

Measures vs Calculated Columns in DAX and Power BI

Jessica Hill

Explore DAX & Power BI differences between measures & calculated columns, and learn when to use each in this informative blog post.
SQLbits 2022 - The Best Bits

SQLbits 2022 - The Best Bits

Barry Smart

This is a summary of the sessions I attended at SQLbits 2022 in London, which is Europe's largest expert led data conference.
How to Create Custom Buttons in Power BI

How to Create Custom Buttons in Power BI

Paul Waller

Explore a step-by-step guide on designing custom buttons in Power BI, covering shape, form, icons, states, actions, and enhancing report design & UI.
A visual approach to demand management and prioritisation

A visual approach to demand management and prioritisation

Barry Smart

Explore a simple, visual approach to prioritisation that aids decision-making and stakeholder engagement.
Evaluation Contexts in DAX - Context Transition

Evaluation Contexts in DAX - Context Transition

Elisenda Gascon

In this third and final part of this series, we learn how CALCULATE performs Context Transition when used inside of a filter context.
Evaluation Contexts in DAX - Relationships

Evaluation Contexts in DAX - Relationships

Elisenda Gascon

After learning about the two different types of evaluation contexts in our previous post, we now talk about table relationships and how these interact with the filter and row contexts to condition the output of our DAX code.
Testing Power BI Reports with the ExecuteQueries REST API

Testing Power BI Reports with the ExecuteQueries REST API

James Broome

Explore DAX queries for scenario-based testing in Power BI reports to ensure data model validity, rule adherence, and security maintenance.
How to Build a Branded Power BI Report Theme

How to Build a Branded Power BI Report Theme

Paul Waller

Explore translating a company's brand into Power BI reports and extending their visual identity to include the Power BI platform.
Evaluation Contexts in DAX - Filter and Row Contexts

Evaluation Contexts in DAX - Filter and Row Contexts

Elisenda Gascon

Explore DAX query language in Power BI: learn about Evaluation Contexts and their impact on code for improved formula results.
Why you should care about the Power BI ExecuteQueries API

Why you should care about the Power BI ExecuteQueries API

James Broome

Explore the benefits of Power BI's new ExecuteQueries REST API for developers in tooling, process, and integrations.
Data is the new soil

Data is the new soil

Barry Smart

Thinking of data as the new soil is useful in highlighting the key elements that enable a successful data and analytics initiative.
How to use Azure Synapse SQL Serverless to connect Data Lake and Power BI

How to use Azure Synapse SQL Serverless to connect Data Lake and Power BI

Carmel Eve

TL;DR - Using Azure Synapse SQL Serverless, you can query Azure Data Lake and populate Power BI reports across multiple workspaces.
How to test Azure Synapse notebooks

How to test Azure Synapse notebooks

James Broome

Explore data with Azure Synapse's interactive Spark notebooks, integrated with Pipelines & monitoring tools. Learn how to add tests for business rule validation.
Do robots dream of counting sheep?

Do robots dream of counting sheep?

Barry Smart

Some of my thoughts inspired whilst helping out on the farm over the weekend. What is the future of work given the increasing presence of machines in our day to day lives? In which situations can AI deliver greatest value? How can we ease the stress of digital transformation on people who are impacted by it?
How to safely reference a nullable activity output in Azure Synapse Pipelines and Azure Data Factory

How to safely reference a nullable activity output in Azure Synapse Pipelines and Azure Data Factory

Ed Freeman

Discover Azure Data Factory's null-safe operator for referencing activity outputs that may not always exist. Learn to use it effectively.
How Azure Synapse unifies your development experience

How Azure Synapse unifies your development experience

Ian Griffiths

Modern analytics requires a multi-faceted approach, which can cause integration headaches. Azure Synapse's Swiss army knife approach can remove a lot of friction.
How do I know if my data solutions are accurate?

How do I know if my data solutions are accurate?

James Broome

Data insights are useless, and even dangerous (as we've seen recently at Public Health England) if they can't be trusted. So, the need to validate business rules and security boundaries within a data solution is critical. This post argues that if you're doing anything serious with data, then you should be taking this seriously.
How to fix the "You need permission to access workspace..." error in Azure Synapse Analytics

How to fix the "You need permission to access workspace..." error in Azure Synapse Analytics

Ed Freeman

Fix the "You need permission" error in Azure Synapse Analytics with this guide, addressing its causes and solutions for Data Engineers/Developers.
How to use the Azure CLI to manage access to Synapse Studio

How to use the Azure CLI to manage access to Synapse Studio

Ed Freeman

Assign roles in Synapse Studio for Azure Synapse Analytics devs using Azure CLI. Accessible by Owners/Contributors of the resource.
The Public Health England Test and Trace Excel error could have been prevented by this one simple step

The Public Health England Test and Trace Excel error could have been prevented by this one simple step

James Broome

Despite the subsequent media reporting, the loss of 16,000 Covid-19 test results at Public Health England wasn't caused by Excel. This post argues that a lack of an appropriate risk and mitigation analysis left the process exposed to human error, which ultimately led to the loss of data and inaccurate reporting. It describes a simple process that could have been applied to prevent the error, and how it will help if you're worried about ensuring quality or reducing risk in your own business, technology or data programmes.
Does Azure Synapse Link redefine the meaning of full stack serverless?

Does Azure Synapse Link redefine the meaning of full stack serverless?

James Broome

Explore Azure Synapse Link for Cosmos DB's impact on 'full stack serverless', No-ETL, and pay-as-you-query analytics.
How to use SQL Notebooks to access Azure Synapse SQL Pools & SQL on demand

How to use SQL Notebooks to access Azure Synapse SQL Pools & SQL on demand

Howard van Rooijen

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.
Deploy an Azure Synapse Analytics workspace using an ARM Template

Deploy an Azure Synapse Analytics workspace using an ARM Template

Ed Freeman

Explore deploying Azure Synapse Analytics workspaces using ARM templates, a popular infrastructure deployment method for organizations.
Azure Synapse Analytics: How serverless is replacing the data warehouse

Azure Synapse Analytics: How serverless is replacing the data warehouse

James Broome

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?
Quick tip – Removing totals from a matrix in Power BI

Quick tip – Removing totals from a matrix in Power BI

Carmel Eve

Here is a quick tip on how to remove column summarisation from a matrix in Power BI!
Quick tip – Updating the sort order of a column in Power BI (avoiding circular references)

Quick tip – Updating the sort order of a column in Power BI (avoiding circular references)

Carmel Eve

Here's a quick tip on how to alter the sort order of a column in Power BI!
How to dynamically choose the correct font colour based on a background colour in Power BI tables.

How to dynamically choose the correct font colour based on a background colour in Power BI tables.

Ed Freeman

Boost Power BI report readability with dynamic font colors for diverse backgrounds, ensuring clear text display and enhanced accessibility.
Benchmarking Azure Synapse Analytics - SQL Serverless, using Polyglot Notebooks

Benchmarking Azure Synapse Analytics - SQL Serverless, using Polyglot Notebooks

James Broome

New Azure Synapse Analytics service offers SQL Serverless for on-demand data lake queries. We tested its potential as a Data Lake Analytics replacement.
Azure Synapse for C# Developers: 5 things you need to know

Azure Synapse for C# Developers: 5 things you need to know

James Broome

Did you know that Azure Synapse has great support for .NET and #csharp? Learning new languages is often a barrier to digital transformation, being able to use existing people, skills, tools and engineering disciplines can be a massive advantage.
5 Reasons why Azure Synapse Analytics should be on your roadmap

5 Reasons why Azure Synapse Analytics should be on your roadmap

James Broome

Explore 5 key reasons to choose Azure Synapse Analytics for your cloud data needs, based on years of experience in driving customer outcomes.
Power BI Embedded: Convention-based dynamic Row-level Security

Power BI Embedded: Convention-based dynamic Row-level Security

Ed Freeman

Explore Power BI Embedded for ISVs, using JavaScript library for personalization, Row-level Security, and modifying Embed Requests for data filtering.
How can I improve my data model in Power BI?

How can I improve my data model in Power BI?

Carmel Eve

Explore how to configure model properties in Power BI for enhanced discoverability and improved data visualisation support.
Why Power BI developers should care about the read/write XMLA endpoint

Why Power BI developers should care about the read/write XMLA endpoint

James Broome

Whilst "read/write XMLA endpoint" might seem like a technical mouthful, its addition to Power BI is a significant milestone in the strategy of bringing Power BI and Analysis Services closer together. As well as closing the gap between IT-managed workloads and self-service BI, it presents a number of new opportunities for Power BI developers in terms of tooling, process and integrations. This post highlights some of the key advantages of this new capability and what they mean for the Power BI developer.
Learning DAX and Power BI – CALCULATE

Learning DAX and Power BI – CALCULATE

Carmel Eve

This is the final blog in a series about DAX and Power BI. This post focuses on the CALCULATE function, which is a unique function in DAX. The CALCULATE function has the ability to alter filter contexts, and therefore can be used to enable extremely powerful and complex processing. This post covers some of the most common scenarios for using CALCULATE, and some of the gotchas in the way in which these different features interact!
Testing Power BI Reports using SpecFlow and .NET

Testing Power BI Reports using SpecFlow and .NET

James Broome

Ensure Power BI report quality by connecting to tabular models, executing scenario-based specs, and validating data, business rules, and security.
Data modelling with Power BI - Loading and shaping data

Data modelling with Power BI - Loading and shaping data

Carmel Eve

Explore data modelling in Power BI, including loading, shaping, and enhancing data. Learn key steps and practical examples.
Learning DAX and Power BI – Related Tables and Relationships

Learning DAX and Power BI – Related Tables and Relationships

Carmel Eve

This is the sixth blog in a series about DAX and Power BI. This post focuses on relationships and related tables. These relationships allow us to build up intricate and powerful models using a combination of sources and tables. The use of relationships in DAX powers many of the features around slicing and page filtering of reports.
Recording of Azure Oxford talk on combatting illegal fishing with Azure (for less than £10/month)

Recording of Azure Oxford talk on combatting illegal fishing with Azure (for less than £10/month)

Carmel Eve

Jess and Carmel recently gave a talk at Azure Oxford on Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month. The recording of that talk is now available for viewing!The talk focuses on the recent work we completed with OceanMind. They run through how to construct a cloud-first architecture based on serverless and data analytics technologies and explore the important principles and challenges in designing this kind of solution. Finally, we see how the architecture we designed through this process not only provides all the benefits of the cloud (reliability, scalability, security), but because of the pay-as-you-go compute model, has a compute cost that we could barely believe!
Testing Power BI Dataflows using SpecFlow and the Common Data Model

Testing Power BI Dataflows using SpecFlow and the Common Data Model

James Broome

Ensure reliable insights with endjin's automated quality gates for validating Power BI Dataflows in complex solutions.
Learning DAX and Power BI – Table Functions

Learning DAX and Power BI – Table Functions

Carmel Eve

This is the fifth blog in a series on DAX and Power BI. This post focuses on table functions. In DAX, table functions return a table which can then be used for future processing. This can be useful if, for example, you want to perform an operation over a filtered dataset. Table functions, like most functions in DAX, operate under the filter context in which they are applied.
Azure Analysis Services - how to save money with automatic shutdown

Azure Analysis Services - how to save money with automatic shutdown

James Broome

Explore Azure Analysis Services for scalable analytics. Control costs via automation with Powershell & Azure DevOps in multi-environment setups.
Learning DAX and Power BI - Aggregators

Learning DAX and Power BI - Aggregators

Carmel Eve

This is the fourth blog in a series about DAX and Power BI. We have so far covered filter and row contexts, and the difference between calculated columns and measures. This post focuses on aggregators. We cover the limitations of the classic aggregators, and demonstrate the power of the iterative versions. We also highlight some of the less intuitive features around how these functions interact with both filter and row contexts.
Power BI Dataflow refresh polling

Power BI Dataflow refresh polling

Ed Freeman

Discover how to programmatically retrieve Power BI Dataflow refresh history for automated operations, enabling efficient dataset refresh management.
Azure Analysis Services: How to update the expression for a calculated column from .NET

Azure Analysis Services: How to update the expression for a calculated column from .NET

James Broome

Learn how to update Azure Analysis Services model schemas in .NET apps using AMO SDK for user-driven analysis.
Wardley Maps - Explaining how OceanMind use Microsoft Azure & AI to combat Illegal Fishing

Wardley Maps - Explaining how OceanMind use Microsoft Azure & AI to combat Illegal Fishing

Jess Panni

Wardley Maps are a fantastic tool to help provide situational awareness, in order to help you make better decisions. We use Wardley Maps to help our customers think about the various benefits and trade-offs that can be made when migrating to the Cloud. In this blog post, Jess Panni demonstrates how we used Wardley Maps to plan the migration of OceanMind to Microsoft Azure, and how the maps highlighted where the core value of their platform was, and how PaaS and Serverless services offered the most value for money for the organisation.
Learning DAX and Power BI – Calculated Columns and Measures

Learning DAX and Power BI – Calculated Columns and Measures

Carmel Eve

Dive into DAX and Power BI in our blog. Learn about calculated columns, measures, and their use in complex visuals.
Optimising C# for a serverless environment

Optimising C# for a serverless environment

Carmel Eve

Tour our OceanMind project using Azure Functions for real-time marine telemetry processing. Learn optimization techniques for serverless environments.
Learning DAX and Power BI – Row Contexts

Learning DAX and Power BI – Row Contexts

Carmel Eve

Here is the second blog in a series around learning DAX and Power BI. This post focuses on row contexts, which are used when iterating over the rows of a table when, for example, evaluating a calculated column. Row contexts along with filter contexts underpin the basis of the DAX language. Once you understand this underlying theory it is purely a case of learning the syntax for the different operations which are built on top of it.
Azure Analysis Services - How to process an asynchronous model refresh from .NET

Azure Analysis Services - How to process an asynchronous model refresh from .NET

James Broome

Learn to use Azure Analysis Services in custom apps via REST API in .NET for efficient async model refreshes.
Learning DAX and Power BI - Filter Contexts

Learning DAX and Power BI - Filter Contexts

Carmel Eve

Explore the basics of DAX and Power BI in this series. This first post focuses on filter contexts, a key concept for effective DAX usage.
Azure Analysis Services: How to execute a DAX query from .NET

Azure Analysis Services: How to execute a DAX query from .NET

James Broome

Explore endless possibilities with dynamic DAX queries in C# for Azure Analysis Services integration in custom apps using the provided code samples.
British Science Week - inspiring the next generation of data scientists

British Science Week - inspiring the next generation of data scientists

James Broome

The theme of this year's British Science Week (6 - 15 March 2020) is "Our Diverse Planet". We'll be getting involved by speaking to school children about the work we've been doing with Oxfordshire-based OceanMind (part of the Microsoft AI for Good programme) to help them combat illegal fishing, hopefully inspiring some of the next generation of data scientists!
Azure Analysis Services - How to query all the measures in a model from .NET

Azure Analysis Services - How to query all the measures in a model from .NET

James Broome

Explore .NET querying methods for integrating Azure Analysis Services beyond data querying into dynamic UIs and APIs.
Azure Analysis Services: How to open a connection from .NET

Azure Analysis Services: How to open a connection from .NET

James Broome

Learn to integrate Azure Analysis Services in apps by establishing server connections. Follow this guide with code samples for essential scenarios.
NDC London 2020 - My highlights

NDC London 2020 - My highlights

Ed Freeman

Ed attended NDC London 2020, along with many of his endjin colleagues. In this post he summarises and reflects upon his favourite sessions of the conference including; "OWASP Top Ten proactive controls" by Jim Manico, "There's an Impostor in this room!" by Angharad Edwards, "How to code music?" by Laura Silvanavičiūtė, "ML and the IoT: Living on the Edge" by Brandon Satrom, "Common API Security Pitfalls" by Philippe De Ryck, and "Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month" by Jess Panni & Carmel Eve.
Azure Analysis Services - integration options using .NET, REST APIs and PowerShell

Azure Analysis Services - integration options using .NET, REST APIs and PowerShell

James Broome

Explore Azure Analysis Services in custom apps using SDKs, PowerShell cmdlets & REST APIs. Learn to choose the right framework in this guide.
Azure Analysis Services: 8 reasons why you might want to integrate into a custom application

Azure Analysis Services: 8 reasons why you might want to integrate into a custom application

James Broome

Explore Azure Analysis Services' versatility in bespoke analysis products & processes, and learn how it can unlock data insights beyond traditional BI.
Building a secure data solution using Azure Data Lake Store (Gen2)

Building a secure data solution using Azure Data Lake Store (Gen2)

Carmel Eve

In this blog we discuss building a secure data solution using Azure Data Lake. Data Lake has many features which enable fine grained security and data separation. It is also built on Azure Storage which enables us to take advantage of all of those features and means that ADLS is still a cost effective storage option!This post runs through some of the great features of ADLS and runs through an example of how we build our solutions using this technology!

Speaking at NDC London: Combatting illegal fishing with Machine Learning and Azure

Carmel Eve

In January 2020, Carmel is speaking about creating high performance geospatial algorithms in C# which can detect suspicious vessel activity, which is used to help alert law enforcement to illegal fishing. The input data is fed from Azure Data Lake Storage Gen 2, and converted into data projections optimised for high-performance computation. This code is then hosted in Azure Functions for cheap, consumption based processing.
Import and export notebooks in Databricks

Import and export notebooks in Databricks

Ed Freeman

Learn to import/export notebooks in Databricks workspaces manually or programmatically, and transfer content between workspaces efficiently.
Demystifying machine learning using neural networks

Demystifying machine learning using neural networks

Carmel Eve

Machine learning often seems like a black box. This post walks through what's actually happening under the covers, in an attempt to de-mystify the process!Neural networks are built up of neurons. In a shallow neural network we have an input layer, a "hidden" layer of neurons, and an output layer. For deep learning, there is simply more hidden layers which allows for combining neuron's inputs and outputs to build up a more detailed picture.If you have an interest in Machine Learning and what is really happening, definitely give this a read (WARNING: Some algebra ahead...)!
How to create a Power BI workspace in an Azure DevOps Pipeline using Powershell

How to create a Power BI workspace in an Azure DevOps Pipeline using Powershell

James Broome

Automate Power BI workspaces creation & security using PowerShell & Azure DevOps pipelines in a wider DevOps strategy with Azure data platform services.
Using Databricks Notebooks to run an ETL process

Using Databricks Notebooks to run an ETL process

Carmel Eve

Explore data analysis & ETL with Databricks Notebooks on Azure. Utilize Spark's unified analytics engine for big data & ML, and integrate with ADF pipelines.
Endjin is a Snowflake Partner

Endjin is a Snowflake Partner

Howard van Rooijen

Snowflake is a cloud native data warehouse platform, that enabled data engineering, data science, data lakes, data sharing and data warehousing. Endjin are very excited to announce our partnership.
Exploring Azure Data Factory - Mapping Data Flows

Exploring Azure Data Factory - Mapping Data Flows

Carmel Eve

Mapping Data Flows are a relatively new feature of ADF. They allow you to visually build up complex data transformation sequences. This can aid in the streamlining of data manipulation and ETL processes, without the need to write any code! This post gives a brief introduction to the technology, and what this could enable!
Snowflake Connector for Azure Data Factory - Part 1

Snowflake Connector for Azure Data Factory - Part 1

Jess Panni

Explore the lack of a native Azure Data Factory connector for Snowflake and discover alternatives for integration between these popular platforms.
Announcing Power BI Weekly!

Announcing Power BI Weekly!

Howard van Rooijen

Power BI has become more successful than anyone anticipated. We've decided to spin out the Power BI category from the Azure Weekly Newsletter into it's own publication - Power BI Weekly.
A conversation about .NET, The Cloud, Data & AI, teaching software engineers and joining endjin with Ian Griffiths

A conversation about .NET, The Cloud, Data & AI, teaching software engineers and joining endjin with Ian Griffiths

Ian Griffiths

When he joined endjin, Technical Fellow Ian sat down with founder Howard for a Q&A session. This was originally published on LinkedIn in 5 parts, but is republished here, in full. Ian talks about his path into computing, some highlights of his career, the evolution of the .NET ecosystem, AI, and the software engineering life.
Thoughts about .NET, The Cloud, AI, ML, and teaching software engineers

Thoughts about .NET, The Cloud, AI, ML, and teaching software engineers

Howard van Rooijen

Ian Griffiths recently joined endjin as a Technical Fellow. We had a long fireside chat, which has been broken down into a 5 part series covering .NET, the Cloud, AI, ML, teaching software engineering, and why he joined endjin.
Branches, builds and modern data pipelines. Let's catch-up!

Branches, builds and modern data pipelines. Let's catch-up!

Ed Freeman

As an apprentice engineer at endjin, you cover a lot of ground, especially at a consultancy which works with the latest and greatest tools to solve its clients' problems. Learn about endjin's Modern Data Platform, which is a culmination of IP, processes and knowledge built from years of implementing high-performance data-driven solutions. Also learn about the types of tools an apprentice gets to use, and the types of things an apprentice learns along the way.
Using CALENDAR vs CALENDARAUTO to generate 'Date' table for year-on-year comparisons

Using CALENDAR vs CALENDARAUTO to generate 'Date' table for year-on-year comparisons

Ed Freeman

Performing Power BI date table generation with CALENDAR & CALENDARAUTO functions. Learn key considerations for time-intelligence calculations in reports.
Using Python inside SQL Server

Using Python inside SQL Server

Ed Freeman

Learn to use SQL Server's Python integration for efficient data handling. Eliminate clunky transfers and easily operationalize Python models/scripts.
Snap Back to Reality – Month 2 & 3 of my Apprenticeship

Snap Back to Reality – Month 2 & 3 of my Apprenticeship

Ed Freeman

Learn what types of things an apprentice gets up to at endjin a few months after joining. You could be learning about Neural Networks: algorithms which mimic the way biological systems process information. You could be attending Microsoft's Future Decoded conference, learning about Bots, CosmosDB, IoT and much more. Hopefully, you wouldn't be in hospital after a ruptured appendix!
How we set up daily Azure spending alerts and saved $10k

How we set up daily Azure spending alerts and saved $10k

Mike Evans-Larah

Read how we set up Azure spending alerts and saved over $10k
My first month as an apprentice at endjin

My first month as an apprentice at endjin

Ed Freeman

Structured apprenticeships provide a great way to build skills whilst getting real-life experience. Endjin's apprenticeship scheme has been refined over years, with an optimal mixture of training, project work, and exposure to commercial processes - a scheme which is designed to build strong foundations for a well-rounded Software Engineering consultant. This post explains the transition from university to an apprenticeship at endjin, including the types of work an apprentice could end up doing, and some examples of real-life learnings from a real-life apprentice.
Creating a PowerBI report with DirectQuery and multiple SQL Database sources using Elastic Query

Creating a PowerBI report with DirectQuery and multiple SQL Database sources using Elastic Query

Alice Waddicor

Learn to build a Power BI dashboard using DirectQuery and ElasticQuery across multiple databases with Alice Waddicor.
Welcome to an internship at endjin!

Welcome to an internship at endjin!

Ed Freeman

A career in software engineering doesn't need to start with a Computer Science degree. The underlying traits of problem solving, a willingness to learn and the ability to collaborate well can be built in any field. Internships provide a great way to get your foot-in-the-door in the professional world, and arm you with some real-life experience for future endeavours. This post describes an internship at endjin, including the type of work you could be asked to do and what you could learn.
AWS vs Azure vs Google Cloud Platform - Mobile Services

AWS vs Azure vs Google Cloud Platform - Mobile Services

Jess Panni

AWS vs Azure vs Google Cloud Platform - Internet of Things

AWS vs Azure vs Google Cloud Platform - Internet of Things

Jess Panni

AWS vs Azure vs Google Cloud Platform - Analytics & Big Data

AWS vs Azure vs Google Cloud Platform - Analytics & Big Data

Jess Panni

AWS vs Azure vs Google Cloud Platform - Database

AWS vs Azure vs Google Cloud Platform - Database

Jess Panni

Querying the Azure DevOps Work Items API directly from Power BI

Querying the Azure DevOps Work Items API directly from Power BI

James Broome

Discover Azure DevOps Work Items features, use RESTful API for insights, and Power BI visualization in our step-by-step guide.
Automating R Unit Tests With Azure DevOps

Automating R Unit Tests With Azure DevOps

Jess Panni

Many organisations are starting to adopt the R Programming Language for their data science and financial modelling scenarios. But just because the language is being used for modelling, doesn't mean you should write unit tests that can be exercised as part of your CI/CD pipeline. In this blog post Jess Panni demonstrates how you can run R unit tests inside Azure DevOps.
TeamCity MetaRunner for creating Release Annotations in Azure Application Insights

TeamCity MetaRunner for creating Release Annotations in Azure Application Insights

Howard van Rooijen

Meta-Runners allow you to easily create reusable build components for TeamCity, in this post I demonstrate how to create a Meta-Runner to create Azure Application Insights Release Annotations.
Automated R Deployments in Azure

Automated R Deployments in Azure

Jess Panni

Machine Learning - the process is the science

Machine Learning - the process is the science

James Broome

What do machine learning and data science actually mean? This post digs into the detail behind the endjin approach to structured experimentation, arguing that the "science" is really all about following the process, allowing you to iterate to insights quickly when there are no guarantees of success.
Embracing Disruption - Financial Services and the Microsoft Cloud

Embracing Disruption - Financial Services and the Microsoft Cloud

Howard van Rooijen

We have produced an insightful booklet called "Embracing Disruption - Financial Services and the Microsoft Cloud" which examines the challenges and opportunities for the Financial Service Industry in the UK, through the lens of Microsoft Azure, Security, Privacy & Data Sovereignty, Data Ingestion, Transformation & Enrichment, Big Compute, Big Data, Insights & Visualisation, Infrastructure, Ops & Support, and the API Economy.
Machine Learning - mad science or a pragmatic process?

Machine Learning - mad science or a pragmatic process?

James Broome

This post looks at what machine learning really is (and isn't), dispelling some of the myths and hype that have emerged as the interest in data science, predictive analytics and machine learning has grown. Without any hard guarantees of success, it argues that machine learning as a discipline is simply trial and error at scale – proving or disproving statistical scenarios through structured experimentation.
Developing U-SQL: Local Data Folder

Developing U-SQL: Local Data Folder

Jess Panni