Today’s organizations need to perform complex analytics to gain insight from data and make data driven decisions. Often this means historical data from various data sources need to reference operational data. This can be a cumbersome task and can put additional load on operational databases that might already be overloaded with the transactional connectivity with live applications.
Mirroring Microsoft Fabric is a modern way to solve this analytics challenge. Microsoft Fabric is meant to be the end-to-end analytics platform for organizations. The One Lake in Microsoft Fabric is a managed storage that can store all historical and analytical data of the organization. With Mirroring, operational databases can be replicated to Lakehouse in Microsoft Fabric. Having data replicated in Microsoft Fabric, makes it easy to use it for analytics and join it with other analytical data sources to perform complex analytics like warehousing or data science.
Mirroring in Microsoft Fabric is currently available for following data sources.
1. Azure Cosmos DB
Any user created databases in Azure Cosmos DB can be replicated to Microsoft Fabric. Sample databases are not replicated in Mirroring. Currently only NoSQL API is available for Mirroring. More on limitations of Azure Cosmos DB Mirroring is available here.
2. Azure SQL
Only user created databases are replicated to Microsoft Fabric. Mirroring needs to be setup on the primary writable Azure SQL DB. Transaction log hold is maintained until the transaction is committed to both main database and Mirrored DB. More limitations of Azure SQL DB Mirroring is available here.
3. Snowflake
Native Snowflake tables are replicated in Mirroring. Tables and Columns with special characters in names are not replicated. More on Snowflake Mirroring limitations can be found here.
Below are advantages of using Mirroring in Microsoft Fabric.
1. Read and Write to Databases and Warehouse from Microsoft Fabric
Since Mirroring is a two-way replication, any writes that are done to Mirrored databases, get committed to the source database. So, Microsoft Fabric can become the single access point to manage all your databases and Warehouses that are setup for Mirroring.
2. Real-Time Data Replication
Microsoft Fabric uses the change data capture feature of source database to replicate data to Mirrored database in real time. When initial Mirroring setup is done, a snapshot of current data is taken and written to the Mirrored database. After that any changes that are committed to the source database, gets reflected in mirrored database in real time.
3. Flexible Analytics
Since the data from source databases are replicated to Microsoft Fabric Lakehouse as Delta Tables, users can use the SQL Analytics Endpoint to perform warehouse queries and reporting as well as use python and spark to perform advanced analytics and build AI models.
4. Cross-Joined Queries
Since Mirrored databases live in Microsoft Fabric One Lake, where you can have all your other analytical data and shortcuts, you can perform join of Mirrored database tables with other Lakehouse and Warehouse tables to perform complex analysis.
5. Power BI Direct Lake mode
Having data in Microsoft Fabric Lakehouse enables Power BI Direct Lake connectivity to get the best possible performance on your Power BI reports.
You can watch my Mirroring in Microsoft Fabric demo on YouTube.