Architecture

Intelligently scheduling cloud data pipelines based on carbon impact can optimize both environmental sustainability and operational efficiency.

Intelligently scheduling cloud data pipelines based on carbon impact can optimize both environmental sustainability and operational efficiency.

Intelligently scheduling cloud data pipelines based on carbon impact can optimize both environmental sustainability and operational efficiency.

Intelligently scheduling cloud data pipelines based on carbon impact can optimize both environmental sustainability and operational efficiency.

No-code/Low-code democratizes software development with little to no coding skills needed. But how do you evaluate if software DIY is the right choice for you?

Explore the benefits of Architecture Decision Records (ADRs) in technical design, with real-world examples applied to different scenarios.

Performing certain deployment operations from a cloud-based CI/CD agent against resources that are only accessible via private networking are problematic. The often-cited solution is to use your own hosted agents that are also connected to the same private networks, however, this introduces additional costs and maintenance overhead. This post discusses an approach that combines the use of cloud-based CI/CD agents with 'just-in-time' allow-listing as an alternative to operating your own private agents.

Why you should make writing unit tests a regular practice and how they will improve your code. This blog explains how to write unit tests in PowerShell using Pester.

Explore Dapr for cloud migration & use Visual Studio container tools to debug containerized services.

Explore Dapr for cloud migration & simplify containerized dev inner-loop using Visual Studio with the Dapr Traffic Control sample application.

If you're interested in the Distributed Application Runtime (Dapr) and how it can simplify migrating existing workloads to use cloud platform services, then this series might be for you. In this post, we continue modifying the excellent Dapr Traffic Control sample application by swapping the MQTT-based messaging with Azure Storage Queues. We will also look at how you can use the Dapr programming model as an alternative to cloud-provider SDKs.

We're constantly using computer networks and the internet, especially as developers. The Web is the most important application on the Internet; this is the second of two posts aiming to explain the basics of the Web.

Explore DAPR's role in migrating workloads to cloud services, focusing on integrating Azure Key Vault for secrets management.

Discover the major application often misidentified as the Internet - the World Wide Web, building on our previous networking post.

Explore the basics of computer networks & internet for developers in this post, simplifying the complex world of networking for better understanding.

Explore Dapr for migrating workloads to cloud services. Learn to enable PaaS Pub-Sub with Azure Service Bus using the Dapr Traffic Control sample app.

Explore Dapr's role in simplifying cloud migration using Azure services, based on the Dapr Traffic Control sample app.

Explore the proxy pattern in this blog post, a design pattern used for controlling access to underlying objects and remote server communication.

This blog is part of a series around design patterns. This post focuses on the facade pattern. The facade pattern provides an interface into a complex system, in order to simplify access to the functionality.

Investigate the decorator pattern in this blog post, a design pattern used to dynamically enhance object functionality.

Despite the subsequent media reporting, the loss of 16,000 Covid-19 test results at Public Health England wasn't caused by Excel. This post argues that a lack of an appropriate risk and mitigation analysis left the process exposed to human error, which ultimately led to the loss of data and inaccurate reporting. It describes a simple process that could have been applied to prevent the error, and how it will help if you're worried about ensuring quality or reducing risk in your own business, technology or data programmes.

Explore Azure Synapse Link for Cosmos DB's impact on 'full stack serverless', No-ETL, and pay-as-you-query analytics.

Explore the adapter pattern in this design patterns series, enabling use of incompatible classes within your solution's interfaces/architecture.

Often when building software we need to outsource part of the solution to a third party. Common examples include authentication, payment, and email, but there are many others. When first designing a solution the decision around which provider to use can be difficult to tackle. At endjin we have developed a system for answering the question "which provider should I use?".

Explore a pattern for async initialization, enabling readonly object construction & leveraging nullable references for fully instantiated classes.

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?

Did you know that Azure Synapse has great support for .NET and #csharp? Learning new languages is often a barrier to digital transformation, being able to use existing people, skills, tools and engineering disciplines can be a massive advantage.

Explore why Microsoft's new Spark offering in Azure Synapse Analytics is a game-changer for Azure Databricks investors.

Jess and Carmel recently gave a talk at Azure Oxford on Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month. The recording of that talk is now available for viewing!The talk focuses on the recent work we completed with OceanMind. They run through how to construct a cloud-first architecture based on serverless and data analytics technologies and explore the important principles and challenges in designing this kind of solution. Finally, we see how the architecture we designed through this process not only provides all the benefits of the cloud (reliability, scalability, security), but because of the pay-as-you-go compute model, has a compute cost that we could barely believe!

Wardley Maps are a fantastic tool to help provide situational awareness, in order to help you make better decisions. We use Wardley Maps to help our customers think about the various benefits and trade-offs that can be made when migrating to the Cloud. In this blog post, Jess Panni demonstrates how we used Wardley Maps to plan the migration of OceanMind to Microsoft Azure, and how the maps highlighted where the core value of their platform was, and how PaaS and Serverless services offered the most value for money for the organisation.

Endjin applies scientific experimentation to software development for validated solutions, using hypotheses for confident designs.

The Azure CNAB Quickstarts Library helps you get up and running with CNAB and Porter. We've built quickstarts covering solutions like WordPress, Ghost, Mattermost, and data platforms like Apache Airflow, SQL Server AlwaysOn clusters and Kubernetes features like an nginx ingress controller and an Azure AD enabled OAuth2 Proxy. We've condensed all our learnings from the past 9 months of working on the project and turned them into a 10 minute video which explains all the key concepts. We hope this video helps accelerate your own CNAB & Porter epiphanies!

In partnership with Microsoft, we have released Azure CNAB Quickstarts Library on GitHub. CNAB (Cloud-Native Application Bundle) is a new specification designed for facilitating the packaging, installation, upgrading and uninstallation of cloud-native solutions in the cloud, on-premise or on the edge. We've created a number of quickstarts covering Apache Airflow, Azure Kubernetes Service, Ghost, Kubeflow, SQL Server Always On and Wordpress to help demonstrate the power of CNAB and Porter.

Discover how Kubernetes, Azure Durable Functions, and Azure API Management transform legacy batch processing code into a RESTful API.

In this post, Ian describes some of the highlights from the NDC London conference

Explore Azure Analysis Services in custom apps using SDKs, PowerShell cmdlets & REST APIs. Learn to choose the right framework in this guide.

In this blog we discuss building a secure data solution using Azure Data Lake. Data Lake has many features which enable fine grained security and data separation. It is also built on Azure Storage which enables us to take advantage of all of those features and means that ADLS is still a cost effective storage option!This post runs through some of the great features of ADLS and runs through an example of how we build our solutions using this technology!

In January 2020, Carmel is speaking about creating high performance geospatial algorithms in C# which can detect suspicious vessel activity, which is used to help alert law enforcement to illegal fishing. The input data is fed from Azure Data Lake Storage Gen 2, and converted into data projections optimised for high-performance computation. This code is then hosted in Azure Functions for cheap, consumption based processing.

Expression Trees are one of .NET's super powers; but they are hugely overlooked and misunderstood. They can be used for on-the-fly code generation; building DSLs, and producing high performance computational algorithms. I'm trying to gather evidence about how you are using them, to make a case for future investments in the Expression Tree subsystem in .NET Core / .NET 5.0

Explore Azure Durable Functions as an alternative to long-running operations, overcoming HTTP Triggers' 230s limit in Azure Load Balancer.

This blog is part of a series around design patterns. This post focuses on the composite pattern. The composite pattern is often used in situations where you want to me able to treat groups and individuals in the same way during processing.

Explore how to build secure solutions on Azure using Azure Functions and Managed Identities, eliminating the need for storing credentials.

Investigate the differences between factory method and abstract factory design patterns in this first blog of our series.

Software architecture and Agile project planning are often seen to be at odds. However, here at endjin we think that the way in which they intersect solves a lot of the common issues surrounding architecture. The key to successful architecture is constantly keeping the drivers in mind and having a tight communication loop as the architecture is implemented. These concepts are ones which are key to agile project management, and the combination of these two disciplines can be extremely powerful and successful when applied correctly.

Creating a architecture in an iterative and incremental solution delivery is challenging. I've evolved an approach akin to Painting By Numbers, and am inspired by a quote from the artist and film maker Dave McKean.

Explore the lack of a native Azure Data Factory connector for Snowflake and discover alternatives for integration between these popular platforms.

At endjin we have a high quality bar when it comes to our code. As part of this we carry out regular code reviews. One of the tools we have used for these code reviews is NDepend. This is the second in a blog series written as we carried out that process. This post focuses on the insight you can quickly gain just by glancing at the NDepend UI.

At endjin we have a high quality bar when it comes to our code. As part of this we carry out regular code reviews. One of the tools we have used for these code reviews is NDepend. This is the first in a blog series written as we carried out that process. This post runs through the different metrics used by NDepend, and the reasons that each of these can be an indication of code quality.

This is the final blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the JOIN operator.

Fix DLL locking errors in Azure Function deployment by using "deploy from package" option. This runs the function from a package file, resolving issues.

Fourth in a series, this blog post explores the inner workings of the Rx WINDOW operator for better Rx understanding.

When he joined endjin, Technical Fellow Ian sat down with founder Howard for a Q&A session. This was originally published on LinkedIn in 5 parts, but is republished here, in full. Ian talks about his path into computing, some highlights of his career, the evolution of the .NET ecosystem, AI, and the software engineering life.

Explore the workings of Rx operators in this series. This first post delves into the WHERE operator for a deeper Rx understanding.

Explore TPL dataflow in this second blog of our series. Learn how this .NET library simplifies parallel data processing with a pipeline model.

This is the first blog in a series about dataflow. The series focuses on TPL dataflow, but this post gives an overview of dataflow as a whole.The crucial thing to understand when using dataflow is that the data is in control. In most conventional programming languages, the programmer determines how and when the code will run. In dataflow, it is the data that drives how the program executes. The movement of data controls the flow of the program.

This is a post about the importance of reducing your dependencies as much as possible when designing solutions.If as much as possible is under your own control, there is far less room for synchronization errors and update conflicts.

Ian Griffiths recently joined endjin as a Technical Fellow. We had a long fireside chat, which has been broken down into a 5 part series covering .NET, the Cloud, AI, ML, teaching software engineering, and why he joined endjin.

Explore the importance of API strategy, governance, security, and design in Digital Transformation projects.

We've been helping customers adopt Microsoft Azure since 2010, we have produced a lot of thought leadership to help people think about the steps required, the risk involved and how to plan a successful adoption.

We've just released an updated version of our Azure Technology Selector poster - helping you to choose the right Azure service, based on your use-case.

Hymans Robertson was set up in Glasgow in 1921 and is one of the longest established independent firms of consultants and actuaries in the UK. Hymans Robertson soon realised that the computational requirements of their models exceeded the capacity of their on-premise datacentres and that the most cost effective solution would be to use the cloud to perform their Big Compute. But before they could harness the cloud to help them solve their Big Data problems, the business needed to understand the ramifications of moving to the cloud; everything from regulatory, risk and compliance concerns,  to how their internal Ops team would need to evolve and adapt, and how to deal with moving data from on-prem into the cloud.

Exploring the creation of a tool to detect accidental .NET API changes for accurate SemVer in NuGet packages.

Consumption based pricing is a one of the USPs of Cloud PaaS services, but the default settings aren't necessarily optimised for cost. Significant savings can be made from understanding your workload.

Explore how to use Azure Automation for running VMs during office hours. Learn to create runbooks for automated tasks at pre-defined times.