Skip to content

Create SQL Serverless views from Azure Synapse Pipelines

· 12 min watch
James Broome By James Broome Director of Engineering

Discover SQL Serverless schema updates in Synapse Pipelines and dynamically manage views for precise data lake queries.

About this talk

Data Toboggan 2022

SQL Serverless allows you to add structured schemas over standard file formats in your data lake storage. For all the benefits this brings, one of the trade offs is that your schema is only enforced "on read", rather than "on write".

So if it's your Synapse Pipelines that are responsible for writing/updating the data that is stored in your data lake, then they should also be responsible for updating the schema of the SQL Serverless views that you use to query it.

In this session, James Broome will show a simple way to dynamically update SQL Serverless views from with a Synapse Pipeline so that they're always up to date with the data in your data lake.

About the presenter

James Broome

Director of Engineering

James Broome

James has spent 20+ years delivering high quality software solutions addressing global business problems, with teams and clients across 3 continents. As Director of Engineering at endjin, he leads the team in providing technology strategy, data insights and engineering support to organisations of all sizes - from disruptive B2C start-ups, to global financial institutions. He's responsible for the success of our customer-facing project delivery, as well as the capability and growth of our delivery team.

Load ?

See our privacy statement for how we handle your data.

Manage cookie preferences

Third-party widgets you have consented to appear below. Revoking stops the widget right away and prevents it from loading on future visits — you'll be asked to consent again next time you click its button.