Simple play icon Course
Skills

Modeling Streaming Data for Processing with Apache Beam

by Janani Ravi

The Apache Beam unified model allows us to process batch as well as streaming data using the same API. Several execution backends such as Google Cloud Dataflow, Apache Spark, and Apache Flink are compatible with Beam.

What you'll learn

Streaming data usually needs to be processed real-time or near real-time which means stream processing systems need to have capabilities that allow them to process data with low latency, high performance and fault-tolerance. In this course, Modeling Streaming Data for Processing with Apache Beam, you will gain the ability to work with streams and use the Beam unified model to build data parallel pipelines. First, you will explore the similarities and differences between batch processing and stream processing. Next, you will discover the Apache Beam APIs which allow one to define pipelines that process batch as well as streaming data. Finally, you will learn how windowing operations can be applied to streaming data. When you are finished with this course, you will have a strong grasp of the models and architectures used with streaming data and be able to work with the Beam unified model to define and run transformations on input streams.

About the author

Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing ... more

Ready to upskill? Get started