Microsoft Fabric
Next generation service lowering barriers to adoption and enabling democratisation of data & analytics
-
Pros
- New generation built on top of proven products (Power BI, Synapse)
- Easy to get started - SaaS model accelerates time to getting underlying infrastructure in place
- Streamlines the end to end process of getting from raw data to actionable insights
- One lake provides unified storage layer for all compute engines
- Embracing open standard (delta) for all data at rest, with optimisation through VORDER
- Single unified user experience
- Integration of OpenAI Co-Pilot into development experience
- Common data & analytics languages supported (Python, SQL)
- Embracing open industry standards such as delta lake
- Strong integration between services
- Introduction of new capabilities
- Opinionated "happy path" architectures to de-risk and accelerate time to value
- New features to boost developer productivity and collaboration
- Strong alignment to Data Mesh principles
-
Cons
- Loss of serverless compute as part of new capacity driven commercial model
- No support for C# in notebooks
- Mapping Data Flows are not supported
- In public preview - not yet ready for production workloads!
- Backlog of important features which are "coming soon"
- Compute engines (Spark and SQL) are not fully unified
- User experience not intuitive for pro devs
Microsoft's 3rd generation data and analytics platform.
Microsoft Fabric is a third generation data and analytics platform, building on strong foundations that have been established by Azure Synapse Analytics and Power BI. It is a SaaS solution that further lowers barriers to adoption.
The following Wardley Map summarizes the evolution from Azure Synapse Analytics and Azure Data Lake (Gen 2) to Microsoft Fabric. A shift right driven by "SaaS-fication" and a shift up by making features more accessible to "citizen analysts":
A persona drive user experience
A unified user experience smooths the boundaries between data engineering, data science, analytics and business intelligence. Opening up opportunities to decentralise and democratise data and analytics at scale.
Familiar functionality carried over
Services that you know and love include Azure Data Factory (Synapse Pipelines), Spark Notebooks, Spark Jobs and the next generation of SQL compute engine. What is exciting is that many of the underlying platforms have been enhanced as part of the "SaaS-ification" process to remove some of the friction experienced in current platforms. For example time to spin up Spark clusters is reduced from minutes to seconds.
The following animation loop illustrates how Microsoft Fabric is changing the landscape by evolving and integrating data and analytics features that we are familiar with in platforms such as Azure Synapse Analytics and Power BI:
Some capabilities are not being carried over
However it is disappointing to see that some Synapse capabilities such as Mapping Dataflows are not supported by Microsoft Fabric. There is also no automated upgrade path to move workloads from Synapse to Fabric. See our Synapse verus Fabric: A Side by Side Comparison for a detailed comparison of Synapse versus Fabric, and some initial insight into migration options.
A new user interface that may take getting used to
Fabric makes it possible to explore data, run experiments, develop pipelines, publish curated data, run analytics, build interactive reports and operationalize solutions within a single web-based UI. Those who are familiar with Synapse Studio will likely take some time to adjust to the Microsoft Fabric user experience. It organises artefacts in a different way based on concepts such as personas, workspaces and domains.
Notebooks on steroids!
We are particularly excited about the enhanced Notebook experience. It feels intuitive, modern and more responsive. It builds on the proven notebook experience for documenting your experiments and orchestrating data pipelines. Adding new features such as comments (similar to Microsoft Word), co-editing and the ability to consume files and managed tables from the Lakehouse by dragging and dropping them into a notebook cell.
"Saas-ification" lowers boundaries to adoption
Setting up a new Microsoft Fabric workspace is a breeze because it is SaaS. Therefore security and sign-on is seamless. Users are able to create a Lakehouses (which can be thought of as "Azure Data Lake Storage as a Service") easily ingest (or consume data without moving it using the new shortcut feature), work with data using the array of different tools and publish data and insights.
By providing a tightly integrated, self-serve data platform we envisage that this will enable the evolution of the wider "data value chain" as illustrated in the Wardley Map below. Here you can see how the evolution of the underlying platform will empower organisations to be more data driven:
AI driven productivity
Another very interesting feature is that Microsoft Fabric is also seeking to leverage Microsoft's investment in OpenAI, with co-pilot "prompt engineering" allowing users to describe their requirements and for the technical artefacts (Python, DAX and Power BI visuals) to be generated automatically.
Huge potential, but not yet ready for production workloads
We love that Microsoft Fabric is seeking to accelerate time to value by reducing the cognitive load that has typically come with earlier generation platforms. No longer is there a need to deploy, integrate and manage Azure Resources to establish a base platform. Furthermore, Microsoft overlays an opinionated view of how to architect common workloads, so you don't have to start with a blank sheet of paper. New concepts such as workspaces and domains will provide the layers of abstraction over artifacts to govern, and protect your analytics ecosytem.
While it's still early days it is clear Microsoft Fabric is another significant step forward for organisations that are seeking to become more data driven and we think this will become more so as the product matures.
Read our blog posts about Microsoft Fabric
Per-Property Rows from JSON in Spark on Microsoft Fabric
SQLbits 2024 - The Best Bits
