Hari Kishore Chaparala: When (Apache) AsterixDB Hit An (Apache) Iceberg

Abstract
Apache Iceberg is an open-source table format with rich data management capabilities, including schema evolution, time travel, and efficient data pruning. It offers a reliable foundation for storing and organizing data in a data lake environment. Iceberg specification allows multiple query engines to safely operate on the same data simultaneously. In this talk, we see how we have introduced Apache AsterixDB to the family of query engines that support Iceberg tabe format specification. Apache AsterixDB is an open-source scalable Big Data Management System (BDMS) targeted to efficiently handle large amounts of semi-structured data. AsterixDB uses Hyracks, a partitioned-parallel platform to perform data-intensive computations and analytics. With AsterixDB’s highly parallel execution capabilities and rich analytics support through SQL++ and flexible data model, querying on external datasets in data lake environments becomes seamless. By integrating AsterixDB with Iceberg, we can leverage Iceberg’s data management features and AsterixDB’s querying features for efficient data lake management and advanced analytics.