Shengquan Ni: IcedTea: Efficient and Responsive Time-Travel Debugging in Dataflow Systems

Abstract: As data analytics grow in popularity, the increasing volume of data and complexity of jobs require users to wait longer to see results, hindering productivity and causing frustration. To address this, we developed an actor-based data processing engine optimized for pipelined execution, featuring a flexible interface for defining control messages. This enables users to seamlessly customize and manage interactions during execution.
While interactive systems help users identify incorrect behavior earlier, the pipelined and distributed nature of execution often leads to non-deterministic behavior, making it difficult to pinpoint the root cause of bugs. To tackle this, we created IcedTea, a time-travel debugger with tuple-based step semantics. IcedTea enables lightweight recording of execution, allowing users to roll back to previous states and step forward to investigate issues effectively.
As data jobs increasingly operate in cloud environments, adapting the system to the cloud is crucial. Challenges such as resource isolation and fault tolerance must be addressed to ensure security and reliability in distributed systems.
In this talk, I will focus on IcedTea, a time-travel debugger that allows users to record a pipelined distributed workflow execution and jump back to specific points to replay the execution step-by-step. Our evaluation demonstrates that IcedTea effectively helps identify state-related bugs with minimal overhead added to the original execution.