Browse our archives by topic…
Python

Azure Synapse Analytics versus Microsoft Fabric: A Side by Side Comparison
In this post we compare Azure Synapse Analytics with Microsoft Fabric to understand how features map from Synapse to Fabric.

Data validation in Python: a look into Pandera and Great Expectations
Implement Python data validation with Pandera & Great Expectations in this comparison of their features and use cases.

How to setup Python, PyEnv & Poetry on Windows
Explore using Python virtual environments & Poetry on Windows for smoother workflows, with a script & guide to enhance your dependency management experience.

How To Implement Continuous Deployment of Python Packages with GitHub Actions
This post demonstrates how to use GitHub Actions to automatically publish updates to your Python package to PyPI, and explores whether this constitutes Continuous Deployment

Customizing Lake Databases in Azure Synapse Analytics
Explore Custom Objects in Lake Databases for user-friendly column names, calculated columns, and pre-defined queries in Azure Synapse Analytics.

How to create a semantic model using Synapse Analytics Database Templates
Explore Azure Synapse Analytics Database Templates and learn to create semantic models in this 2nd blog of the series.

Continuous Integration with GitHub Actions
This post gives an overview of Continuous Integrations and shows how you can implement it with GitHub Actions, with an accompanying example Python project

How to apply behaviour driven development to data and analytics projects
In this blog we demonstrate how the Gherkin specification can be adapted to enable BDD to be applied to data engineering use cases.

What is the Shared Metadata Model in Azure Synapse Analytics, and why should I use it?
A lesser known feature of Azure Synapse is the "Shared Metadata Model". Synapse has the capability to automatically synchronize tables created via Synapse Spark with objects you can query via the usual SQL Serverless endpoint, without any additional configuration. This article brings attention to this capability, highlighting the benefits and tradeoffs vs rolling your own SQL Serverless VIEWs.

Extract insights from tag lists using Python Pandas and Power BI
We often come across spreadsheets and CSV files that contain features which are a list of tags or categories. This blog article walks through a simple way of extracting insights from this data using a combination of Pandas for data wrangling and Power BI for analytics and visualisation.

Introduction to Containers and Docker
Explore containerisation & Docker for app development & deployment. Learn to create containerised applications with examples in this intro guide.

How to test Azure Synapse notebooks
Explore data with Azure Synapse's interactive Spark notebooks, integrated with Pipelines & monitoring tools. Learn how to add tests for business rule validation.

How Azure Synapse unifies your development experience
Modern analytics requires a multi-faceted approach, which can cause integration headaches. Azure Synapse's Swiss army knife approach can remove a lot of friction.

How to use SQL Notebooks to access Azure Synapse SQL Pools & SQL on demand
Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

Using Databricks Notebooks to run an ETL process
Explore data analysis & ETL with Databricks Notebooks on Azure. Utilize Spark's unified analytics engine for big data & ML, and integrate with ADF pipelines.

Using Python inside SQL Server
Do you have a bunch of data in SQL Server that you're using ODBC/JDBC to pull data to work with in Python? Using SQL Server's Python integration, you can connect to a SQL Server instance within your preferred IDE and perform the computations on the SQL Server Machine. No more clunky data transferring. Operationalizing a Python model/script is as easy as calling a stored procedure. Any application that can speak to SQL Server can invoke the Python code and retrieve the results. Easy! This blog will provide a few, simple examples which make use of this capability to carry out some simple Python commands, so you can get up and running as quickly as possible.