Skip to content
How to fix the "You need permission to access workspace..." error in Azure Synapse Analytics

How to fix the "You need permission to access workspace..." error in Azure Synapse Analytics

Ed Freeman

Data Engineers/Developers want to get access to Azure Synapse Analytics as quickly as possible to start designing and creating their data solutions. Being denied access to Synapse Studio can be frustrating and slows matters down. This article will address the "You need permission to access workspace..." error, discuss what causes it, and describe how to fix it.
How to use the Azure CLI to manage access to Synapse Studio

How to use the Azure CLI to manage access to Synapse Studio

Ed Freeman

Azure Synapse Analytics developers need to be assigned a role within Synapse Studio in order to access the GUI. The Azure CLI provides one way of programmatically achieving this, which can be done by any Owner or Contributor of the Azure Synapse Analytics resource.
Does Azure Synapse Link redefine the meaning of full stack serverless?

Does Azure Synapse Link redefine the meaning of full stack serverless?

James Broome

Azure Synapse Link for Cosmos DB is a game-changing piece in the Synapse suite of services - extending the support for SQL on Demand to enable querying over the Cosmos DB Analytical Store. This post explores whether the term 'full stack serverless' should now be extended to cover No-ETL and pay-as-you-query analytics, alongside serverless application architectures.
How to use SQL Notebooks to access Azure Synapse SQL Pools & SQL on demand

How to use SQL Notebooks to access Azure Synapse SQL Pools & SQL on demand

Howard van Rooijen

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.
ArrayPool vs MemoryPool—minimizing allocations in AIS.NET

ArrayPool vs MemoryPool—minimizing allocations in AIS.NET

Ian Griffiths

Tracking down unexpected allocations in a high-performance .NET parsing library.
A simple invite user flow for AAD B2C (without custom policies)

A simple invite user flow for AAD B2C (without custom policies)

James Broome

Azure AD B2C is easy to integrate and configure for simple sign-in / sign-up authentication. But there's a step change in complexity as soon as you want to do something outside of the built-in user flows. An "invite user" flow is one such example, but it's also a fairly common requirement in any business or team orientated SaaS application, which makes AAD B2C as less attractive choice. This post describes an alternative approach for this authentication feature that avoids the need for AAD B2C custom policies, allowing you to keep authentication as simple as possible.
Deploy an Azure Synapse Analytics workspace using an ARM Template

Deploy an Azure Synapse Analytics workspace using an ARM Template

Ed Freeman

Azure Synapse Analytics is Microsoft's new unified cloud analytics platform, which will surely be playing a big part in many organizations' technology stacks in the near future. For many organizations, Azure Resource Manager (ARM) templates are the infrastructure deployment method of choice. This blog explains how to deploy an Azure Synapse Analytics workspace using an ARM template.
Using Azure Key Vault for Encryption in C# - A Simple Tutorial

Using Azure Key Vault for Encryption in C# - A Simple Tutorial

Ed Freeman

Do you need to encrypt a piece of data in your application? Do you want Azure Key Vault to secure the key? Well, using C# along with a couple of libraries from the Azure SDK, it couldn't be easier to get up and running. This blog explains how to do just that.
How to prepare for Exam AZ-204: Developing Solutions for Microsoft Azure

How to prepare for Exam AZ-204: Developing Solutions for Microsoft Azure

Carmel Eve

Carmel recently passed the 'Az-204 - Developing Solutions for Microsoft Azure' exam. Here she shares how she went about preparing!
Azure Synapse Analytics: How serverless is replacing the data warehouse

Azure Synapse Analytics: How serverless is replacing the data warehouse

James Broome

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?
Talking about Azure Synapse on Microsoft Mechanics!

Talking about Azure Synapse on Microsoft Mechanics!

Jess Panni

I was recently invited on to Microsoft Mechanics to talk about the new on-demand SQL Serverless offering within Azure Synapse. If you have been following along with my previous blog posts you will know that we've been hard at work applying Azure Synapse against real customer workloads. In the video I take you through the service by solving a real-world IoT problem for one of our telco customers.
Benchmarking Azure Synapse Analytics - SQL Serverless, using .NET Interactive

Benchmarking Azure Synapse Analytics - SQL Serverless, using .NET Interactive

James Broome

There is a new service in town that promises to transform the way you query the contents of your data lake. Azure Synapse Analytics comes with a new offering called SQL Serverless allowing you to query your data on-demand with no need for pre-provisioned resources.When we heard about the new service we were keen to get involved, so for the last 10 months we've been working with the SQL Serverless product group to provide feedback on the service and to help ensure it meets our customers needs. During this time we've put it through it's paces by implementing a range of real-world use cases. We were particularly interested to see how it stacked up as a replacement for Data Lake Analytics, where to date there has been no clear and easy migration path.
Azure Synapse for C# Developers: 5 things you need to know

Azure Synapse for C# Developers: 5 things you need to know

James Broome

Did you know that Azure Synapse has great support for .NET and #csharp? Learning new languages is often a barrier to digital transformation, being able to use existing people, skills, tools and engineering disciplines can be a massive advantage.
Using multiple azure-cli credentials within automation

Using multiple azure-cli credentials within automation

James Dawson

Have you ever needed an automated process to use alternative credentials for a subset of tasks? This post will demonstrate a technique that allows you setup multiple, concurrent authenticated sessions when using the azure-cli and switch freely between them.
Does Azure Synapse Analytics spell the end for Azure Databricks?

Does Azure Synapse Analytics spell the end for Azure Databricks?

James Broome

Have you or are you about to invest in Azure Databricks? If so, the new Spark offering in Azure Synapse Analytics is likely to have grabbed your attention and rightly so. Why is Microsoft putting yet another Spark offering on the table and what does it mean for you?
5 Reasons why Azure Synapse Analytics should be on your roadmap

5 Reasons why Azure Synapse Analytics should be on your roadmap

James Broome

For years we have been building modern cloud data solutions on Azure and helping our customers transform their use of data to drive outcomes. Here are 5 reasons why Azure Synapse Analytics might just be the service that we have been crying out for.
Why Power BI developers should care about the new read/write XMLA endpoint

Why Power BI developers should care about the new read/write XMLA endpoint

James Broome

Whilst "read/write XMLA endpoint" might seem like a technical mouthful, its addition to Power BI is a significant milestone in the strategy of bringing Power BI and Analysis Services closer together. As well as closing the gap between IT-managed workloads and self-service BI, it presents a number of new opportunities for Power BI developers in terms of tooling, process and integrations. This post highlights some of the key advantages of this new capability and what they mean for the Power BI developer.
How does Azure Key Vault help me secure my data?

How does Azure Key Vault help me secure my data?

Carmel Eve

Azure Key Vault is used to protect encryption keys and secrets. These keys and secrets can be used to access encrypted data and protected services. Individual Key Vaults can be used to preserve security information for isolating keys and secrets. The keys stored can be either hardware or software protected. Access to the keys and secrets is controlled using Azure Active Directory, RBAC and access policies.
Testing Power BI Reports using SpecFlow and .NET

Testing Power BI Reports using SpecFlow and .NET

James Broome

Despite being inherently difficult to test, the need to validate data modelling, business rules and security boundaries in Power BI reports is important, as well as the need for ensuring that quality doesn't regress over time as the insights evolve. This post explains that, by connecting to the underlying tabular model, it is possible to execute scenario-based specifications to add quality gates and build confidence in Power BI reports, just as any other software project.
Recording of Azure Oxford talk on combatting illegal fishing with Azure (for less than £10/month)

Recording of Azure Oxford talk on combatting illegal fishing with Azure (for less than £10/month)

Carmel Eve

Jess and Carmel recently gave a talk at Azure Oxford on "Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month). The recording of that talk is now available for viewing!The talk focuses on the recent work we completed with OceanMind. They run through how to construct a cloud-first architecture based on serverless and data analytics technologies and explore the important principles and challenges in designing this kind of solution. Finally, we see how the architecture we designed through this process not only provides all the benefits of the cloud (reliability, scalability, security), but because of the pay-as-you-go compute model, has a compute cost that we could barely believe!
Testing Power BI Dataflows using SpecFlow and the Common Data Model

Testing Power BI Dataflows using SpecFlow and the Common Data Model

James Broome

Whilst testing Power BI Dataflows isn't something that many people think about, it's critical that business rules and associated data preparation steps are validated to ensure the right insights are available to the right people across the organisation. Data insights are useless, even dangerous, if they can't be trusted, so despite the lack of "official support" or recommended approaches from Microsoft, endjin treat Power BI solutions just as any other software project with respect to testing - building automated quality gates into the end to end development process. This post outlines an approach that endjin has used to test Power BI Dataflows to add quality gates and build confidence in large and complex Power BI solutions.
Azure Analysis Services - how to save money with automatic shutdown

Azure Analysis Services - how to save money with automatic shutdown

James Broome

Azure Analysis Services provides an enterprise-grade analytical platform with massive scale and flexibility. But, as one of the more expensive services in the Azure platform, consideration should be given to cost management, especially in multi-environment ALM scenarios. This post explains how to massively reduce running costs through automation using Powershell and orchestration tools like Azure DevOps.
Building a proximity detection pipeline

Building a proximity detection pipeline

Carmel Eve

At endjin, our approach focuses on using scientific experimental method to support the creation of fully proved and tested decision making, and the use of scientific research to support our work. This post runs through how we applied that process to creation a pipeline to detect vessel proximity.This is an example which is based around the project we recently worked on with OceanMind. In this project we helped them to build a #serverless architecture which could detect vessel proximity in close to real time. The vessel proximity events we detected were then fed into machine learning algorithms in order to detect illegal fishing!Carmel also runs through some of the actual calculations we used to detect proximity, how we used #data projections to efficiently process large quantitities of incoming data, and the use of #durablefunctions to orchestrate the processing.
Azure Analysis Services: How to update the expression for a calculated column from .NET

Azure Analysis Services: How to update the expression for a calculated column from .NET

James Broome

This post explains how to update Azure Analysis Services model schemas from inside custom .NET applications. Whilst not a common scenario for most, it shows that this is easy to do using the AMO SDK. So, there's nothing stopping you from developing complex and rich end-user functionality over the top of your data analysis solutions - providing run-time, user-driven schema changes like "what if" analysis.
Wardley Maps - Explaining how OceanMind use Microsoft Azure & AI to combat Illegal Fishing

Wardley Maps - Explaining how OceanMind use Microsoft Azure & AI to combat Illegal Fishing

Jess Panni

Wardley Maps are a fantastic tool to help provide situational awareness, in order to help you make better decisions. We use Wardley Maps to help our customers think about the various benefits and trade-offs that can be made when migrating to the Cloud. In this blog post, Jess Panni demonstrates how we used Wardley Maps to plan the migration of OceanMind to Microsoft Azure, and how the maps highlighted where the core value of their platform was, and how PaaS and Serverless services offered the most value for money for the organisation.
Integration Testing Azure Functions with SpecFlow and C#, Part 5 - Using Corvus.SpecFlow.Extensions in a build pipeline

Integration Testing Azure Functions with SpecFlow and C#, Part 5 - Using Corvus.SpecFlow.Extensions in a build pipeline

Jonathan George

If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. Even now, several years after their introduction, the testing story for Functions is not hugely well defined. In the final post in this series, we show how to ensure specs written using Corvus.SpecFlow.Extensions can run as part of your build pipeline.
Integration Testing Azure Functions with SpecFlow and C#, Part 4 - Controlling your functions with additional configuration

Integration Testing Azure Functions with SpecFlow and C#, Part 4 - Controlling your functions with additional configuration

Jonathan George

If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. Even now, several years after their introduction, the testing story for Functions is not hugely well defined. In the fourth of this series of posts, we look at how configuration can be supplied from your tests to the functions apps being tested.
Integration Testing Azure Functions with SpecFlow and C#, Part 3 - Using hooks to start Functions

Integration Testing Azure Functions with SpecFlow and C#, Part 3 - Using hooks to start Functions

Jonathan George

If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. Even now, several years after their introduction, the testing story for Functions is not hugely well defined. In the third of a series of posts, we look at using classes in the Corvus.SpecFlow.Extensions library to run functions apps via scenario and feature hooks.
Integration Testing Azure Functions with SpecFlow and C#, Part 2 - Using step bindings to start Functions

Integration Testing Azure Functions with SpecFlow and C#, Part 2 - Using step bindings to start Functions

Jonathan George

If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. Even now, several years after their introduction, the testing story for Functions is not hugely well defined. In the second of a series of posts, we look at using step bindings provided by the Corvus.SpecFlow.Extensions library to run functions apps as part of your SpecFlow scenarios.
Integration Testing Azure Functions with SpecFlow and C#, Part 1 - Introduction

Integration Testing Azure Functions with SpecFlow and C#, Part 1 - Introduction

Jonathan George

If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. Even now, several years after their introduction, the testing story for Functions is not hugely well defined. In the first of a series of posts, we look at some different approaches to testing your functions apps, and introduce the Corvus.SpecFlow.Extensions library.
Azure Analysis Services - How to process an asynchronous model refresh from .NET

Azure Analysis Services - How to process an asynchronous model refresh from .NET

James Broome

Integrating Azure Analysis Services into custom applications doesn't just mean read-only data querying. But if your application changes the underlying model, it will need to be re-processed before the changes take effect. This post describes how to use the REST API for Azure Analysis Services inside a custom .NET application to perform asynchronous model refreshes, meaning your applications can reliably and efficiently deal with model updates.
Introducing Ais.Net - High-Performance Parsing in C#

Introducing Ais.Net - High-Performance Parsing in C#

Ian Griffiths

As part of our work with OceanMind, endjin wrote a high performance .NET AIS parser. AIS (Automatic Identification System) is how commercial ships report location information. This blog describes the parser, and the performance techniques it uses.
Azure Analysis Services: How to execute a DAX query from .NET

Azure Analysis Services: How to execute a DAX query from .NET

James Broome

Being able to construct DAX queries dynamically in C# means the possibilities are endless in terms of integrating Azure Analysis Services queries into your custom applications, and with the code samples in this post, you have everything you need to get started.
British Science Week - inspiring the next generation of data scientists

British Science Week - inspiring the next generation of data scientists

James Broome

The theme of this year's British Science Week (6 - 15 March 2020) is "Our Diverse Planet". We'll be getting involved by speaking to school children about the work we've been doing with Oxfordshire-based OceanMind (part of the Microsoft AI for Good programme) to help them combat illegal fishing, hopefully inspiring some of the next generation of data scientists!
How can I contribute to the Azure CNAB Quickstarts Library?

How can I contribute to the Azure CNAB Quickstarts Library?

Howard van Rooijen

The Azure CNAB Quickstart Templates we've created are only half the story. Much of the work we've done over the last few months involved the authoring, contribution and DevOps pipelines required to support an open source project. The project is inspired by the original Azure Quickstart Templates - which over the last 5 years has grown to over 850 templates. In this post we're going to explain how you can author CNAB templates and contribute them.
Setting up Porter on Windows

Setting up Porter on Windows

Mike Larah

Porter is a tool based on the CNAB (Cloud Native Application Bundle) spec. It can be used for building, managing, and installing application bundles. This guide will walk you through how to get set up with Porter on Windows.
Introducing the Azure CNAB Quickstarts Library

Introducing the Azure CNAB Quickstarts Library

Mike Larah

In partnership with Microsoft, we have released Azure CNAB Quickstarts Library on GitHub. CNAB (Cloud-Native Application Bundle) is a new specification designed for facilitating the packaging, installation, upgrading and uninstallation of cloud-native solutions in the cloud, on-premise or on the edge. We've created a number of quickstarts covering Apache Airflow, Azure Kubernetes Service, Ghost, Kubeflow, SQL Server Always On and Wordpress to help demonstrate the power of CNAB and Porter.
Exposing legacy batch processing code online using Azure Durable Functions, API Management and Kubernetes

Exposing legacy batch processing code online using Azure Durable Functions, API Management and Kubernetes

Jonathan George

In this post we show how a combination of Kubernetes, Azure Durable Functions and Azure API Management can be used to make legacy batch processing code available as a RESTful API. This is a great example of how serverless technologies can be used to expose legacy software to the public internet in a controlled way, allowing you to reap some of the benefits of a cloud first approach without fully rewriting and migrating existing software.
Azure Analysis Services - How to query all the measures in a model from .NET

Azure Analysis Services - How to query all the measures in a model from .NET

James Broome

Integrating Azure Analysis Services into custom applications means more than just querying the data. By surfacing the metadata in your models, you can build dynamic and customisable UIs and APIs, tailored to the needs of the client application. This post explains how easy it is to query model metadata from .NET, so you can create deeper integrations between your data insights and your custom applications.
Azure Analysis Services: How to open a connection from .NET

Azure Analysis Services: How to open a connection from .NET

James Broome

One of the first steps in integrating Azure Analysis Services into your applications is creating and opening a connection to the server - just like any other database technology. This post explains the ins and outs of creating Azure Analysis Services connections, including code samples for each of the key scenarios.
NDC London 2020 - My highlights

NDC London 2020 - My highlights

Ed Freeman

Ed attended NDC London 2020, along with many of his endjin colleagues. In this post he summarises and reflects upon his favourite sessions of the conference including; "OWASP Top Ten proactive controls" by Jim Manico, "There's an Impostor in this room!" by Angharad Edwards, "How to code music?" by Laura Silvanavičiūtė, "ML and the IoT: Living on the Edge" by Brandon Satrom, "Common API Security Pitfalls" by Philippe De Ryck, and "Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month" by Jess Panni & Carmel Eve.
NDC London Day 3 Retrospective - from personal projects to developer comedy

NDC London Day 3 Retrospective - from personal projects to developer comedy

Jonathan George

Along with several of my endjin colleagues, I attended NDC London in January this year - here's a run through of the sessions I attended on Day 3 and my thoughts. This final day was a mixed bag, taking in talks on drumming and AKKA.net, as well as something a bit more close to home - a session from endjin's own Jess Panni and Carmel Eve on our recent project for OceanMind.
NDC London Day 2 Retrospective - Full Stack, from SPAs to high performance .NET Core via Websockets

NDC London Day 2 Retrospective - Full Stack, from SPAs to high performance .NET Core via Websockets

Jonathan George

Along with several of my endjin colleagues, I attended NDC London in January this year - here's a run through of the sessions I attended on Day 2 and my thoughts. This day was UI heavy, with sessions on Vuejs and UI testing, but I also learned more about GraphQL and writing high performance C# code.
NDC London Day 1 Retrospective - dangerous AI, microservices, diagnostics and Blazor

NDC London Day 1 Retrospective - dangerous AI, microservices, diagnostics and Blazor

Jonathan George

A retrospective of my first day at NDC London 2020, taking in sessions on AI and Machine Learning, Capability Mapping, Micro Frontends, Diagnostics and Blazor.
Azure Analysis Services - integration options using .NET, REST APIs and PowerShell

Azure Analysis Services - integration options using .NET, REST APIs and PowerShell

James Broome

With a variety of integration support through client SDKs, PowerShell cmdlets and REST APIs, it can be hard to know where to start with integrating Azure Analysis Services into your custom applications. This posts walks through the options, and lays out a simple guide to choosing the right framework.
Azure Analysis Services: 8 reasons why you might want to integrate into a custom application

Azure Analysis Services: 8 reasons why you might want to integrate into a custom application

James Broome

We've done a lot of work at endjin with Azure Analysis Services over the last couple of years - but none of it has been what you'd call "traditional BI". We've pulled, twisted and bent it in all sorts of directions, using it's raw analytical processing power to underpin bespoke analysis products and processes. This post explains some of the common (and not-so-common) reasons why you might want to do similar things, and how Azure Analysis Services might be the key to unlocking your data insights.
AI for Good Hackathon

AI for Good Hackathon

Ian Griffiths

Towards the end of last year, Microsoft invited endjin along to a hackathon session they hosted at the IET in London as part of their AI for Good initiative. I've been thinking about the event and the broader work Microsoft is doing here a lot lately, because it gets to the heart of what I love about working in this industry: computers can magnify our power to do to good.
Building a secure data solution using Azure Data Lake Store (Gen2)

Building a secure data solution using Azure Data Lake Store (Gen2)

Carmel Eve

In this blog from the Azure Advent Calendar 2019 we discuss building a secure data solution using Azure Data Lake. Data Lake has many features which enable fine grained security and data separation. It is also built on Azure Storage which enables us to take advantage of all of those features and means that ADLS is still a cost effective storage option!This post runs through some of the great features of ADLS and runs through an example of how we build our solutions using this technology!

Speaking at NDC London: Combatting illegal fishing with Machine Learning and Azure

Carmel Eve

In January 2020, Carmel is speaking about creating high performance geospatial algorithms in C# which can detect suspicious vessel activity, which is used to help alert law enforcement to illegal fishing. The input data is fed from Azure Data Lake Storage Gen 2, and converted into data projections optimised for high-performance computation. This code is then hosted in Azure Functions for cheap, consumption based processing.
Troubleshooting "NotImplemented / Access is denied" error when integrating Azure App Service with Azure Virtual Network (VNet)

Troubleshooting "NotImplemented / Access is denied" error when integrating Azure App Service with Azure Virtual Network (VNet)

Ed Freeman

Are you configuring your Azure App Service to use a VNet? The regional VNet integration for an Azure App Service is a preview feature, and so comes with some quirks. One of the documented limitations of this preview feature is that "The feature is only available from newer App Service scale units that support PremiumV2 App Service plans." This has interesting implications as to how you need to deploy your App Service, otherwise you could end up with a rather perplexing pattern of errors. Read about how we came about this error, and how we ended up fixing it.
Comparing GitHub Actions and Azure Pipelines: High-level concepts

Comparing GitHub Actions and Azure Pipelines: High-level concepts

Mike Larah

GitHub Actions is GitHub's new CI/CD platform. It is comparable with Azure Pipelines, which forms part of the Azure DevOps suite. In this post, Mike Larah looks at the similarities and differences in the high-level concepts and terminology between the two platforms.
Long Running Functions in Azure Data Factory

Long Running Functions in Azure Data Factory

Jess Panni

While on first inspection Azure Function look like a good candidate for long running operations as they can run for 10 minutes on a Consumption plan, HTTP Triggers only run for 230 seconds because of a limitation with the Azure Load Balancer. In this blog post Jess Panni demonstrates how Durable Functions can be used instead.
How Azure DevTestLabs is helping me climb Everest

How Azure DevTestLabs is helping me climb Everest

Carmel Eve

Remote working allows us to work from anywhere we want. This brings a huge amount of flexibility in freedom, however we do need the help of a working laptop! When Carmel's laptop gave in just before a trip, she used Azure DevTestLabs to allow her to continue to work using a 10 year old Mac that probably couldn't wouldn't have been up to the task alone...
Running Azure functions in Docker on a Raspberry Pi 4

Running Azure functions in Docker on a Raspberry Pi 4

Jonathan George

For one of my first experiments with the Raspberry Pi 4, I decided to get an Azure Function running in a Docker container. This post gives a step-by-step guide on how to do it, as well as providing code you can use a starting point for your own experiments.
Secure function-to-function authentication in Azure without the need for credentials

Secure function-to-function authentication in Azure without the need for credentials

Carmel Eve

Building a secure solution on Azure can be a daunting task. Using Azure Functions and Managed Identities, we have built up a pattern for giving services access to one another, woithout the need to store credentials. These managed identities can be given access to necessary resources. For example, they can be granted roles and added to access control lists in ADLS Gen2 accounts, or the ability to access keys in key vault. This means that data can be securely accessed without needing to store connection strings or app passwords.
How to use the AzureAD module in PowerShell Core

How to use the AzureAD module in PowerShell Core

James Broome

Whilst some of the Azure Active Directory PowerShell for Graph module (AzureAD) functionality has been rolled into the new Azure PowerShell Az module, it's not currently (and might never be) a replacement for the full power of what you can achieve with AzureAD. So, there's every chance you'll find yourself needing to use both side-by-side. This post explains how to do that using the new cross-platform PowerShell Core.
How to create a Power BI workspace in an Azure DevOps Pipeline using Powershell

How to create a Power BI workspace in an Azure DevOps Pipeline using Powershell

James Broome

A Power BI based solution typically consists of a variety of technologies - for example Azure data platform services containing source data. As such, automation of Power BI resources needs to be considered as part of a wider DevOps strategy. This post describes the specific steps needed in order to fully automate the creation and security of Power BI workspaces using Powershell and Azure DevOps pipelines.
Using Databricks Notebooks to run an ETL process

Using Databricks Notebooks to run an ETL process

Carmel Eve

Here at endjin we've done a lot of work around data analysis and ETL. As part of this we have done some work with Databricks Notebooks on Microsoft Azure. Notebooks can be used for complex and powerful data analysis using Spark. Spark is a "unified analytics engine for big data and machine learning". It allows you to run data analysis workloads, and can be accessed via many APIs. This means that you can build up data processes and models using a language you feel comfortable with. They can also be run as an activity in a ADF pipeline, and combined with Mapping Data Flows to build up a complex ETL process which can be run via ADF.
Exploring Azure Data Factory - Mapping Data Flows

Exploring Azure Data Factory - Mapping Data Flows

Carmel Eve

Mapping Data Flows are a relatively new feature of ADF. They allow you to visually build up complex data transformation sequences. This can aid in the streamlining of data manipulation and ETL processes, without the need to write any code! This post gives a brief introduction to the technology, and what this could enable!
Snowflake Connector for Azure Data Factory - Part 2

Snowflake Connector for Azure Data Factory - Part 2

Jess Panni

Snowflake Connector for Azure Data Factory - Part 1

Snowflake Connector for Azure Data Factory - Part 1

Jess Panni

Enforce resource tagging with Azure Policy

Enforce resource tagging with Azure Policy

Mike Larah

This blog post details how we used Azure Policy to enforce Azure resources were tagged with appropiate tags and ensured tags were inherited from parent resource groups where possible.
Avoiding deployment locking errors by running Web and Functions Apps from packages

Avoiding deployment locking errors by running Web and Functions Apps from packages

Carmel Eve

This post walks through the fix for DLL locking errors when trying to deploy an Azure Function. The solution was to switch over to the new "deploy from package" option when deploying the functions. This fixes the file locking problem because instead of deploying the DLLs, the function will run from a package file added to its directory.
How to run a script on an existing Azure Virtual Machine

How to run a script on an existing Azure Virtual Machine

Mike Larah

Using PowerShell and the VM Custom Script Extension to download and execute scripts on existing Azure VMs
Managing applications using Azure AD, service principals and managed identities: A permissions story

Managing applications using Azure AD, service principals and managed identities: A permissions story

Carmel Eve

The complexities around Azure Active Directory can be difficult to understand. This post runs through some of the key concepts - AAD apps, service principles, managed identities, and walks through an example of how to set some of this up!
Cosmos DB - Request Units charged for processing a Gremlin API request

Cosmos DB - Request Units charged for processing a Gremlin API request

Howard van Rooijen

Thoughts about .NET, The Cloud, AI, ML, and teaching software engineers

Thoughts about .NET, The Cloud, AI, ML, and teaching software engineers

Howard van Rooijen

OpenAPI Document Converters for Visual Studio 2017

OpenAPI Document Converters for Visual Studio 2017

Howard van Rooijen

How we set up daily Azure spending alerts and saved $10k

How we set up daily Azure spending alerts and saved $10k

Mike Larah

Kickstart your API proposition with the API Maturity Assessment

Kickstart your API proposition with the API Maturity Assessment

Howard van Rooijen

Rest APIs underpin many Digital Transformation strategies. As with all software projects the devil is in the detail; your API strategy, governance, security, design principals are as, if not more imporant than building a working API.
How to plan your cloud transformation journey

How to plan your cloud transformation journey

Howard van Rooijen

Azure Tech Selector - 2017 Edition

Azure Tech Selector - 2017 Edition

Matthew Adams

Best of the Recorded Azure Sessions at BUILD 2017

Best of the Recorded Azure Sessions at BUILD 2017

Howard van Rooijen

Creating a PowerBI report with DirectQuery and multiple SQL Database sources using Elastic Query

Creating a PowerBI report with DirectQuery and multiple SQL Database sources using Elastic Query

Alice Waddicor

Sometimes you want to build a Power BI dashboard that pulls in data from two different data sources. In this blog post Alice Waddicor demonstrates how you can use DirectQuery and multiple databased via ElasticQuery.
Benchmarking the Cloud against on-premise data centres

Benchmarking the Cloud against on-premise data centres

Howard van Rooijen

Advanced Azure Resource Manager template patterns – t-shirt sizing and optional resources

Advanced Azure Resource Manager template patterns – t-shirt sizing and optional resources

Richard Kerslake

One very useful but little used pattern when working with Resource Manager templates, is the ability to use parameters to optionally deploy resources, constrain certain resource configurations based on other user defined parameters, or to toggle parameters based on other values. This blog post explores some of these options.
Automating office security with Synology, Surveillance Station, OneDrive and Power Automate

Automating office security with Synology, Surveillance Station, OneDrive and Power Automate

Mike Larah

Read about how we set up automated backups of our office security camera footage and used Power Automate to alert us if anything went wrong
AWS vs Azure vs Google Cloud Platform - Networking

AWS vs Azure vs Google Cloud Platform - Networking

Jess Panni

How to use Power BI Embedded with AspNetCore

How to use Power BI Embedded with AspNetCore

Matthew Adams

AWS vs Azure vs Google Cloud Platform - Mobile Services

AWS vs Azure vs Google Cloud Platform - Mobile Services

Jess Panni

AWS vs Azure vs Google Cloud Platform - Internet of Things

AWS vs Azure vs Google Cloud Platform - Internet of Things

Jess Panni

AWS vs Azure vs Google Cloud Platform - Analytics & Big Data

AWS vs Azure vs Google Cloud Platform - Analytics & Big Data

Jess Panni

AWS vs Azure vs Google Cloud Platform - Database

AWS vs Azure vs Google Cloud Platform - Database

Jess Panni

AWS vs Azure vs Google Cloud Platform - Storage & Content Delivery

AWS vs Azure vs Google Cloud Platform - Storage & Content Delivery

Jess Panni

AWS vs Azure vs Google Cloud Platform - Compute

AWS vs Azure vs Google Cloud Platform - Compute

Jess Panni

AWS vs Azure vs Google Cloud Platform

AWS vs Azure vs Google Cloud Platform

Jess Panni

Cloud Adoption: A Deep Dive into the Swiss Cheese Model

Cloud Adoption: A Deep Dive into the Swiss Cheese Model

Howard van Rooijen

Automating creation of new ALM environments using PowerShell and Azure DevOps

Automating creation of new ALM environments using PowerShell and Azure DevOps

Richard Kerslake

Did you know that you can fully automate the bootstrapping process of setting up an Azure DevOps enviornment? This post shows you how.
"But it works on my cloud!" - are your developers still making the same mistakes in a world of DevOps and PaaS services?

"But it works on my cloud!" - are your developers still making the same mistakes in a world of DevOps and PaaS services?

James Broome

In the world of DevOps, cloud and platform services, how does a developer's "definition of done" need to change? This post argues that as the silos of development and operations are broken down, the responsibility of understanding the whole solution increases meaning, to truly take advantage of the cloud, the need for quality and professionalism is critical for success.
Deploying to Azure using Azure Resource Manager templates and Octopus Deploy

Deploying to Azure using Azure Resource Manager templates and Octopus Deploy

Richard Kerslake

Learn how to combine the power of Azure Resource Manager and Octopus Deploy for a frictionless Azure DevOps experience.
Cloud Adoption: Risks & Mitigations Analysis

Cloud Adoption: Risks & Mitigations Analysis

Howard van Rooijen

Using Postman to load test an Azure Machine Learning web service

Using Postman to load test an Azure Machine Learning web service

Richard Kerslake

Azure Machine Learning Studio is a fantastic service for experimentation, but you can also easily productionize your machine learning models. In this post we show how to create an Azure ML Studio web service and test it using Postman.
TeamCity MetaRunner for creating Release Annotations in Azure Application Insights

TeamCity MetaRunner for creating Release Annotations in Azure Application Insights

Howard van Rooijen

Automated R Deployments in Azure

Automated R Deployments in Azure

Jess Panni

Using Azure Automation to run VMs during office hours only – using graphical runbooks

Using Azure Automation to run VMs during office hours only – using graphical runbooks

Richard Kerslake

We've improved our approach for saving money by turning off our virtual machines outside of office hours. This post explored how to do it using graphical runbooks.
Embracing Disruption - Financial Services and the Microsoft Cloud

Embracing Disruption - Financial Services and the Microsoft Cloud

Howard van Rooijen

We have produced an insightful booklet called "Embracing Disruption - Financial Services and the Microsoft Cloud" which examines the challenges and opportunities for the Financial Service Industry in the UK, through the lens of Microsoft Azure, Security, Privacy & Data Sovereignty, Data Ingestion, Transformation & Enrichment, Big Compute, Big Data, Insights & Visualisation, Infrastructure, Ops & Support, and the API Economy.
The 100 Year Start-up: Embracing Disruption in Financial Services

The 100 Year Start-up: Embracing Disruption in Financial Services

Howard van Rooijen

Hymans Robertson was set up in Glasgow in 1921 and is one of the longest established independent firms of consultants and actuaries in the UK. Hymans Robertson soon realised that the computational requirements of their models exceeded the capacity of their on-premise datacentres and that the most cost effective solution would be to use the cloud to perform their Big Compute. But before they could harness the cloud to help them solve their Big Data problems, the business needed to understand the ramifications of moving to the cloud; everything from regulatory, risk and compliance concerns,  to how their internal Ops team would need to evolve and adapt, and how to deal with moving data from on-prem into the cloud.
Why is blockchain revolutionising Financial Services?

Why is blockchain revolutionising Financial Services?

Matthew Adams

There is a lot of hype about the blockchain - usually wrapped up with talk about Bitcoin and crypto-currencies. In this article, we look at its impact on trust, and auditability in financial services, and why it may (or may not) be appropriate for your solutions.
Regulatory Compliance and Cloud Adoption

Regulatory Compliance and Cloud Adoption

Matthew Adams

In this post we review the FCA's guidelines for the adoption of cloud services by FinTech businesses, and help you to understand their impact across the value chain.
FinTech Week and the Microsoft Cloud

FinTech Week and the Microsoft Cloud

Matthew Adams

Azure Resource Manager authentication from a PowerShell script

Azure Resource Manager authentication from a PowerShell script

Richard Kerslake

Learn how to authenticate against Azure Resource Manager using PowerShell, for fully automated deployments.
Azure Weekly turns 1 & we're celebrating by giving away 5 Windows IoT Raspberry Pi 2 starter kits

Azure Weekly turns 1 & we're celebrating by giving away 5 Windows IoT Raspberry Pi 2 starter kits

Howard van Rooijen

Azure Batch - Time is Money in Big Compute

Azure Batch - Time is Money in Big Compute

James Broome

Azure Machine Learning–experimenting with training data proportions using the SMOTE module

Azure Machine Learning–experimenting with training data proportions using the SMOTE module

Alice Waddicor

Using Azure Resource Manager and PowerShell DSC to create and provision a VM

Using Azure Resource Manager and PowerShell DSC to create and provision a VM

Richard Kerslake

Azure Resource Manager and PowerShell Desired State Configuration are powerful tools for provisioning and then configuring your virutal machines in Azure. This post delves into the details.
Spinning up 16,000 A1 Virtual Machines on Azure Batch

Spinning up 16,000 A1 Virtual Machines on Azure Batch

Howard van Rooijen

Azure data services part 3: Azure Machine Learning

Azure data services part 3: Azure Machine Learning

Alice Waddicor

Azure data services part 2: Stream Insight

Azure data services part 2: Stream Insight

Alice Waddicor

Azure data services part 1: HDInsight

Azure data services part 1: HDInsight

Alice Waddicor

SpecFlow Extensions for Azure Storage Emulator

SpecFlow Extensions for Azure Storage Emulator

Howard van Rooijen

Speed kills: decrease page load time and increase conversions with Azure CDN

Speed kills: decrease page load time and increase conversions with Azure CDN

Pascal Arnould

A CDN, short for Content Delivery Network, is essentially a network of servers set-up, in various geographic locations, with the aim of speeding up the delivery of content to end-users across the Internet.
Azure Automation scheduler and daylight saving time

Azure Automation scheduler and daylight saving time

Richard Kerslake

Everything in Azure runs in Coordinated Universal Time; learn how to deal with Daylight saving time in Azure Automation, so your jobs still run on time.
Visualise your Azure Table Storage data with Power BI

Visualise your Azure Table Storage data with Power BI

Alice Waddicor

Azure Web Apps: Package Restore against a private NuGet Feed

Azure Web Apps: Package Restore against a private NuGet Feed

Howard van Rooijen

Using a different New Relic application name per environment with Azure Cloud Services

Using a different New Relic application name per environment with Azure Cloud Services

Richard Kerslake

Just because you're using the cloud, doesn't mean you don't use traditional DTAP environments. But you do need to configure your cloud services to understand them.
Reporting errors to New Relic from an Azure Worker Role

Reporting errors to New Relic from an Azure Worker Role

Richard Kerslake

In this post we demonstrate how to use New Relic to monitor an Azure Cloud Service Worker Role, step by step.
A step by step guide to choosing the right Azure technology for your solution

A step by step guide to choosing the right Azure technology for your solution

Matthew Adams

To help guide the client through the selection process (and to act as an aide memoire when we come to try and remember why particular choices were made), we created a handy flow-chart to take you through the technology landscape, and suggest some recommended-practice solutions.
Generating and using a certificate to authorise Azure Automation

Generating and using a certificate to authorise Azure Automation

Alice Waddicor

Diagnostic logging with Azure SDK 2.5

Diagnostic logging with Azure SDK 2.5

Richard Kerslake

Learn how to use diagnostics in Azure Virtual Machines and Cloud Service Web or Worker Roles using the Azure SDK.
Using Azure Automation to run VMs during office hours only

Using Azure Automation to run VMs during office hours only

Richard Kerslake

One of the strongest selling points of cloud computing is that you only pay for what you use. Running virtual machines can become expensive, but you can save significant money if you turn those VMs off outside of office hours.
Elasticsearch gotchas and tips - part 2

Elasticsearch gotchas and tips - part 2

Richard Kerslake

We've delivered a number of Elasticsearch solutions on Azure. In this second post in the series, we share some useful tips.
Have you signed up for the Azure Weekly newsletter?

Have you signed up for the Azure Weekly newsletter?

Howard van Rooijen

Azure Weekly Newsletter

Azure Weekly Newsletter

Matthew Adams

Troubleshooting Twilio with New Relic

Troubleshooting Twilio with New Relic

Richard Kerslake

We created an an automated, telephony-based candidate interviewing system using .NET and Azure. We used New Relic and Twilio to simplify first line support. In this post we delve into some of the technical details.
Restrict access to Azure Websites by whitelisting

Restrict access to Azure Websites by whitelisting

Mike Larah

By utilising the IP and Domain Restrictions feature in IIS, it is possible to lock down your Azure Website to only allow access to IP addresses and domains that you have specified in a whitelist. Learn how in this post.
Gotchas when installing an Elasticsearch cluster on Microsoft Azure

Gotchas when installing an Elasticsearch cluster on Microsoft Azure

Richard Kerslake

We've built a number of Elasticsearch clusters on Microsoft Azure. This blog post contains some of our top tips and gotchas.

Azure Tip: Cannot connect to Virtual Machine after changing Virtual Machine Size

Howard van Rooijen

Configuring TeamCity and YouTrack to use Office 365 for Email Notifications

Howard van Rooijen

Downloading Windows Azure Subscription Files

Pascal Arnould

If you use Azure Management Studio, you can quickly setup your connections by importing your publish settings...

Azure From The Trenches: Updating Service Configuration Whilst Running In The Emulator

Mike Larah

Azure Cloud Migration Portal

Matthew Adams

RavenDB in the Cloud

Matthew Adams