At present marks an necessary milestone for open requirements and the broader knowledge neighborhood. We’re working with Microsoft to combine the info managed each in Unity Catalog and OneLake utilizing open requirements. The Unity Catalog Open API specification gives an interface to alternate knowledge each methods, zero copy, to open up entry to your complete lakehouse ecosystem for engines in both platform.
For almost a decade, Databricks and Microsoft have partnered to assist organizations unlock the worth of their knowledge by way of the openness and adaptability of the lakehouse. At present, that partnership takes one other step ahead, giving prospects extra alternative in how they entry and govern knowledge throughout platforms.
The Path to Open Interoperability
As open desk codecs like Delta Lake and Iceberg gained adoption, new silos emerged on the catalog-level. To handle this, Databricks launched Unity Catalog Open APIs, establishing an open normal for unified governance throughout knowledge and AI, whereas additionally absolutely supporting the Iceberg REST catalog API for tabular knowledge. This dual-API method empowers organizations to undertake best-in-class options and keep away from vendor lock-in whereas seamlessly sharing knowledge throughout many knowledge platforms and SaaS Functions. We additionally launched catalog federation to offer a single ruled view throughout all knowledge platforms. This mannequin, open APIs powering catalog federation, has develop into the business normal for interoperability.
At present, we’re excited to see this open imaginative and prescient validated by our companions at Microsoft.
Microsoft provides help for Unity Catalog Open APIs
As we introduced with our companions at Microsoft right this moment, our collaboration is deepening.
Microsoft has already added help for studying from Azure Databricks with OneLake Mirroring, permitting Cloth engines to entry the entire knowledge managed in Unity Catalog seamlessly. In contrast to different OneLake integrations with proprietary platforms, this makes all knowledge in Unity Catalog – together with the best efficiency tables – accessible in OneLake with out copying, altering code, or extra prices.
At present, Microsoft and Databricks are extending their openness by integrating much more natively. With Microsoft OneLake including help for the Unity Catalog Open API normal, Azure Databricks customers will have the ability to seamlessly learn knowledge from OneLake.
With OneLake including help for open catalog APIs together with Iceberg REST and Unity Catalog Open APIs, Microsoft is making OneLake structured and unstructured knowledge from companies like Dynamics, Dataverse, Blob retailer, and so on. natively accessible to your complete open lakehouse ecosystem, together with Databricks.
Databricks Builds on the Open Basis
Leveraging this expanded functionality, Databricks is including help to make all OneLake knowledge seamlessly accessible in Unity Catalog.
This permits zero-copy studying of OneLake knowledge from inside Databricks guaranteeing seamless interoperability and entry throughout each platforms, giving Databricks prospects entry to all their knowledge within the Microsoft ecosystem to energy their most superior AI, machine studying, and large-scale knowledge engineering workloads.
Catalog Integration
- Studying from Unity Catalog in Cloth – Typically Accessible Now
Microsoft has launched Azure Databricks mirroring. Clients can mirror Azure Databricks knowledge into OneLake, by way of Unity Catalog guaranteeing that ALL knowledge managed by Azure Databricks is immediately accessible throughout Microsoft Cloth workloads. Each platforms can work over the identical copy of knowledge with no knowledge motion. - Studying from OneLake in Azure Databricks – EOY 2025
This new functionality, allows Azure Databricks to learn knowledge from OneLake by way of their Unity Catalog Open APIs and permits your entire OneLake knowledge to be leveraged seamlessly in Databricks. Clients can create Unity Catalog Connections to OneLake primarily based on Managed Identities from EntraID, and might seamlessly mount knowledge gadgets from any Cloth workspace, straight as a catalog in Unity.
Connecting to OneLake knowledge in Azure Databricks
The Future is Open
This milestone displays how far the business has come towards an open, interoperable knowledge ecosystem. Clients not want to decide on between innovation and interoperability—they will have each.
With Microsoft now embracing the open catalog normal, we’re excited to proceed constructing the way forward for knowledge and AI on this basis. Keep tuned for extra bulletins and roadmap updates, together with help for Azure Databricks writing knowledge straight in OneLake, as we deepen our integration.
