Python

This blog explores how to implement robust validation strategies within the medallion architecture using Pandera, helping you catch issues early and maintain clean, trustworthy data.

Learn how to configure VS Code to enable stepping into external code when debugging a Python Behave test

Spark doesn't always interpret JSON how we'd like. For example, if each key/value pair in a JSON object is conceptually one item, Spark won't give you a row per item by default. This article shows how to nudge Spark in the right direction.

The first step on the road to implementing observability in your Python notebooks is basic logging. In this post, we look at how you can use Python's built in logging inside a Synapse notebook.

In this Microsoft Fabric vs Synapse comparison we examine how features map from Azure Synapse to Fabric.

Implement Python data validation with Pandera & Great Expectations in this comparison of their features and use cases.

Explore using Python virtual environments & Poetry on Windows for smoother workflows, with a script & guide to enhance your dependency management experience.

Discover using GitHub Actions for auto-updates to Python packages on PyPI, assessing its role in Continuous Deployment.

Explore Custom Objects in Lake Databases for user-friendly column names, calculated columns, and pre-defined queries in Azure Synapse Analytics.

Explore Azure Synapse Analytics Database Templates and learn to create semantic models in this 2nd blog of the series.

This post gives an overview of Continuous Integrations and shows how you can implement it with GitHub Actions, with an accompanying example Python project

In this blog we demonstrate how the Gherkin specification can be adapted to enable BDD to be applied to data engineering use cases.

Explore Azure Synapse's 'Shared Metadata Model' feature. Learn how it syncs Spark tables with SQL Serverless, its benefits, and tradeoffs.

Discover how to extract insights from spreadsheets and CSV files using Pandas and Power BI in this blog post.

Explore containerisation & Docker for app development & deployment. Learn to create containerised applications with examples in this intro guide.

Explore data with Azure Synapse's interactive Spark notebooks, integrated with Pipelines & monitoring tools. Learn how to add tests for business rule validation.

Modern analytics requires a multi-faceted approach, which can cause integration headaches. Azure Synapse's Swiss army knife approach can remove a lot of friction.

Wishing Azure Synapse Analytics had support for SQL notebooks? Fear not, it's easy to take advantage rich interactive notebooks for SQL Pools and SQL on Demand.

Explore data analysis & ETL with Databricks Notebooks on Azure. Utilize Spark's unified analytics engine for big data & ML, and integrate with ADF pipelines.

Learn to use SQL Server's Python integration for efficient data handling. Eliminate clunky transfers and easily operationalize Python models/scripts.