Data Mesh and Lakehouse - Matei Zaharia, Databricks

Ғылым және технология

Data mesh is an emerging data management architecture that makes it easier for organizations to collaborate. At the same time, in the technology layer beneath it, a new class of data management systems have been gaining traction: lakehouse systems that combine the capabilities of data warehouses and data lakes, bringing high performance query processing and governance to large-scale data stored in open formats. These systems, such as Delta Lake and Apache Hudi, are letting organizations simplify their data architectures by not having to manage separate data warehouses that load subsets of their data to achieve governance or performance. In addition, through new open source projects like Delta Sharing, these systems can directly support data mesh architectures not just within the same technology platform, but across clouds and vendors. I’ll talk about some of the recent work to support these patterns in the Delta Lake open source project and how data mesh and lakehouse systems can combine to create dramatically simpler data architectures.

Пікірлер: 2

  • @baatchus1519
    @baatchus15192 жыл бұрын

    Great talk from Matei ! Databricks with its open data lakehouse platform is a perfect fit with Data Mesh. Lakehouse ❤️ Data Mesh

  • @AlexanderSergeenko
    @AlexanderSergeenko Жыл бұрын

    I appreciate the great talk!

Келесі