How to actually make architectural decisions for a Data Platform

SQLBits 2024

Are you drowning in a sea of architectural options for your new data platform? Are there people in your organisation with strong opinions on which tools should be used, and for what? Are you unsure how to prove/evidence your gut feeling as to how things should be built? Well - this session's for you!

If you're in charge of architecting a new data platform, it can be difficult to make concrete decisions that pushes you closer to your end goal - building a successful solution. This session will introduce a tried-and-tested suite of strategies and tools that will help you efficiently navigate the unforgiving task of architecting a modern data platform.

We'll take a look at artifacts such as ADRs (Architectural Decision Records), NFR (Non-functional requirements) documents, and discuss strategies inspired from the Agile methodology (such as failing fast) that can help you in your endeavour.

Chapters

• 00:00 Introduction and Background
• 00:30 The Problem with Too Many Options
• 01:38 Understanding Platform Requirements
• 04:20 Organizational Factors and Conway's Law
• 07:53 Evaluating Products and Vendors
• 10:13 The Evaluation Process
• 15:05 Making the Final Decision
• 16:39 Closing Tips and Final Thoughts