Microsoft Fabric
Next generation service lowering barriers to adoption and enables democratisation of data & analytics
-
Pros
- New generation built on top of proven products (Power BI, Synapse)
- Easy to get started - SaaS model accelerates time to getting underlying infrastructure in place
- Streamlines the end to end process of getting from raw data to actionable insights
- OneLake provides unified storage layer for all compute engines
- Embracing open standard (delta) for all data at rest, with optimisation through VORDER
- Single unified user experience
- Integration of OpenAI Co-Pilot into development experience
- Common data & analytics languages supported (Python, SQL)
- Embracing open industry standards such as delta lake
- Strong integration between services
- Introduction of new novel capabilities such as Activator, Python Notebooks
- Opinionated "happy path" architectures to de-risk and accelerate time to value
- New features to boost developer productivity and collaboration
- Strong alignment with Data Mesh principles
-
Cons
- Loss of serverless compute as part of new capacity driven commercial model
- No support for C# in notebooks
- Azure Synapse Mapping Data Flows are not supported
- Backlog of important features which are "coming soon"
- Compute engines (Spark and SQL) are not fully unified
- User experience not intuitive for pro devs, although some work has been done to address this feedback!
- DevOps features still in development
Microsoft's 3rd generation data and analytics platform.
Microsoft Fabric is a third generation data and analytics platform, building on strong foundations that have been established by Azure Synapse Analytics and Power BI. It is a SaaS solution that further lowers barriers to adoption.
The following Wardley Map summarizes the evolution from Azure Synapse Analytics and Azure Data Lake (Gen 2) to Microsoft Fabric. A shift right driven by "SaaS-fication" and a shift up by making features more accessible to "citizen analysts":
A persona drive user experience
A unified user experience smooths the boundaries between data engineering, data science, analytics and business intelligence. Opening up opportunities to decentralise and democratise data and analytics at scale.
Familiar functionality carried over
Services that you know and love include Azure Data Factory (Synapse Pipelines), Spark Notebooks, Spark Jobs and the next generation of SQL compute engine. What is exciting is that many of the underlying platforms have been enhanced as part of the "SaaS-ification" process to remove some of the friction experienced in current platforms. For example time to spin up Spark clusters is reduced from minutes to seconds.
The following animation loop illustrates how Microsoft Fabric is changing the landscape by evolving and integrating data and analytics features that we are familiar with in platforms such as Azure Synapse Analytics and Power BI:
Some capabilities are not being carried over
However it is disappointing to see that some Synapse capabilities such as Mapping Dataflows are not supported by Microsoft Fabric. There is also no automated upgrade path to move workloads from Synapse to Fabric. See our Synapse versus Fabric: A Side by Side Comparison for a detailed comparison of Synapse versus Fabric, and some initial insight into migration options.
A new user interface that may take getting used to
Fabric makes it possible to explore data, run experiments, develop pipelines, publish curated data, run analytics, build interactive reports and operationalize solutions within a single web-based UI. Those who are familiar with Synapse Studio will likely take some time to adjust to the Microsoft Fabric user experience. It organises artefacts in a different way based on concepts such as personas, workspaces and domains.
Notebooks on steroids!
We are particularly excited about the enhanced Notebook experience. It feels intuitive, modern and more responsive. It builds on the proven notebook experience for documenting your experiments and orchestrating data pipelines. Adding new features such as comments (similar to Microsoft Word), co-editing and the ability to consume files and managed tables from the Lakehouse by dragging and dropping them into a notebook cell.
The recent addition of Python Notebooks which is pre-installed with packages such as Polars and DuckDB is an existing development. This new features reflects that Spark is overkill for many workloads. Python Notebooks run on a single node, they are therefore quicker to spin up, offer a more snappy developer experience and are more cost effective.
"Saas-ification" lowers boundaries to adoption
Setting up a new Microsoft Fabric workspace is a breeze because it is SaaS. Therefore security and sign-on is seamless. Users are able to create a Lakehouses (which can be thought of as "Azure Data Lake Storage as a Service") easily ingest (or consume data without moving it using the new shortcut feature), work with data using the array of different tools and publish data and insights.
By providing a tightly integrated, self-serve data platform we envisage that this will enable the evolution of the wider "data value chain" as illustrated in the Wardley Map below. Here you can see how the evolution of the underlying platform will empower organisations to be more data driven:
AI driven productivity
Another advantage that Fabric offers over Synapse is that it integrates with Microsoft's investment in OpenAI, with a co-pilot experience that allow users to describe their requirements and for the technical artefacts (Python, DAX and Power BI visuals) to be generated automatically, or for the Copilot to help with code reviews, completion and resolving errors.
Huge potential, ready for targeted use
We love that Microsoft Fabric is seeking to accelerate time to value by reducing the cognitive load that has typically come with earlier generation platforms. No longer is there a need to deploy, integrate and manage Azure Resources to establish a base platform. Furthermore, Microsoft overlays an opinionated view of how to architect common workloads, so you don't have to start with a blank sheet of paper. New concepts such as workspaces and domains will provide the layers of abstraction over artifacts to govern, and protect your analytics ecosystem.
The platform continues to move at pace with new features constantly being released. Keeping up with the pace of change can be daunting. The level of change means that some features are more mature than others - the platform is no so large, that it merits its own technology radar to enable each of the major components to be tracked independently.
Microsoft Fabric should be considered as a strategic option for organisations that are seeking to become more data driven, the product has matured significantly since it became generally available, we would therefore recommend that you trial it in a targeted way to gain direct experience implementing real world use cases on the platform. Use this experience as a feedback loop to make further informed decisions about ramping up adoption.
Read our blog posts about Microsoft Fabric
DuckDB in Practice: Enterprise Integration and Architectural Patterns
Working locally with spark dev containers
