BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Information Systems Group - ECPv6.4.0.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://isg.ics.uci.edu
X-WR-CALDESC:Events for Information Systems Group
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20180311T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20181104T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20181116T150000
DTEND;TZID=America/Los_Angeles:20181116T160000
DTSTAMP:20261108T043214
CREATED:20181222T021505Z
LAST-MODIFIED:20181222T021505Z
UID:842-1542380400-1542384000@isg.ics.uci.edu
SUMMARY:Dr. Andrey Balmin and Mayank Pradhan (Workday): Workday Prism Analytics: Unifying Interactive and Batch Data Processing Using Apache Spark
DESCRIPTION:Abstract: Workday Prism Analytics enables data discovery and interactive Business Intelligence analysis for Workday customers. To prepare data for analysis\, business users can setup data transformation pipelines in an interactive\, self-service\, modern data prep environment. Thus\, Workday Prism Analytics needs to run three types of scalable data processing applications: “always on” query engine and data prep applications\, and on-demand batch execution of transformation pipelines. We standardized on Apache Spark and Spark SQL for all three applications\, due to its scalability\, as well as\, flexibility and extensibility of the Spark’s Catalyst compiler. All applications share much of the compilation and execution code\, except for sampling\, caching\, and result extraction. \n  \nIn this talk we will\, first\, introduce Workday and then Prism Analytics. We will then zoom into Spark-based interactive and batch data processing components of Prism Analytics. We will then describe the data prep transformations\, and their compilation into Spark DataFrames\, through Spark-SQL Catalyst plans\, in both interactive and batch mode. We will focus on some challenges we encountered while compiling and executing complex pipelines and queries. For example\, Spark SQL compilation times exceeded execution time for some low-latency queries. And compiled plans grew dangerously for data prep pipelines with multiple self-joins and self-unions. We will describe caching\, sampling\, and query compilation techniques that allow us to support interactive user experience. This includes a join co-sampling component that improves system usability when joining large datasets. Finally\, we will conclude with an overview of the open challenges that we plan to tackle in the future. \n  \nBios: \nDr. Andrey Balmin is a Sr. Principal Engineer at Workday\, where he is building the self-service Prism Analytics platform\, continuing the work he began at Platfora (which was acquired by Workday in 2016). Prior to this\, he was a Research Staff Member at IBM Almaden Research Center where he focused on search and query processing of semi-structured and graph-structured data in Data Warehousing and\, later\, Big Data platforms. He holds a Ph.D. degree in Computer Science from UC San Diego. \n  \nMayank Pradhan is a Senior Engineering Manager at Workday focused on building Analytics Platform for Workday’s customers. His team develops the backend engines in cloud that run a variety of workloads including\, data ingestion\, interactive data preparation\, and OLAP cubes for large scale\, high volume interactive querying. Prior to Workday\, Mayank worked on industry shifting data processing products like Platfora\, ParAccel (the columnar database technology behind Redshift)\, IBM DB2. He has 18 years of industry experience building distributed databases. He did his M.S in Computer Science from Santa Clara University and BS Computer Science from Pune University India.
URL:https://isg.ics.uci.edu/event/dr-andrey-balmin-and-mayank-pradhan-workday-workday-prism-analytics-unifying-interactive-and-batch-data-processing-using-apache-spark/
LOCATION:DBH 3011
END:VEVENT
END:VCALENDAR