Streamlined MapReduce: Massively Parallel Processing of Data Streams

Critical applications affect human lives, their safety and their privacy. The navigation of emergency services or fire trucks would be efficient if traffic jams are avoided. Proactive disaster control would be possible with automated traffic surveillance. Several critical applications need an infrastructure that provides efficient processing of real-time data, which enables the provisioning of useful pieces of information in real-time. The first step into building such an infrastructure is to provide for the massively parallel processing of streamed data, which is the core of this book. In this book, we describe the design and implementation of a stream-based distributed processing system for continuous queries. Inspired by Google's MapReduce programming model running on Google File System, we build a distributed stream system and an in-memory MapReduce runtime environment to enable developers post their continuous queries on data streams to be processed in real time.