Featured resource
Tech Upskilling Playbook 2025
Tech Upskilling Playbook

Build future-ready tech teams and hit key business milestones with seven proven plays from industry leaders.

Learn more
  • Path icon Learning Path
  • Libraries: This path is only available in the libraries listed. To access this path, purchase a license for the corresponding library.
  • AI
  • Cloud
  • Data

Data Engineering on Google Cloud

10 Courses
14 Hours
Skill IQ

This path provides participants a hands-on introduction to designing and building data processing systems on Google Cloud Platform. Through a combination of presentations, demos, and hand-on labs, participants will learn how to design data processing systems, build end-to-end data pipelines, analyze data and derive insights. The courses cover structured, unstructured, and streaming data.

Content in this path

Intermediate

This section opens with the two key components of any data pipeline, which are data lakes and warehouses. The first course highlights use-cases for each type of storage and dives into the available data lake and warehouse solutions on Google Cloud Platform in technical detail. Also, the course describes the role of a data engineer, the benefits of a successful data pipeline to business operations, and examines why data engineering should be done in a cloud environment. Data pipelines typically fall under one of the Extra-Load, Extract-Load-Transform or Extract-Transform-Load paradigms. Hence, the second course in this section describes which paradigm should be used and when for batch data. Furthermore, this course covers several technologies on Google Cloud Platform for data transformation including BigQuery, executing Spark on Cloud Dataproc, pipeline graphs in Cloud Data Fusion and serverless data processing with Cloud Dataflow.

Advanced

This section covers two things: (ii) Processing streaming data, which is becoming increasingly popular as streaming enables businesses to get real-time metrics on business operations, and (ii) Incorporating machine learning into data pipelines increases the ability of businesses to extract insights from their data. The first course covers how to build streaming data pipelines on Google Cloud Platform. Cloud Pub/Sub is described for handling incoming streaming data. The course also covers how to apply aggregations and transformations to streaming data using Cloud Dataflow, and how to store processed records to BigQuery or Cloud Bigtable for analysis. The second course covers several ways machine learning can be included in data pipelines on Google Cloud Platform depending on the level of customization required. For little to no customization, this course covers AutoML. For more tailored machine learning capabilities, this course introduces AI Platform Notebooks and BigQuery Machine Learning. Also, this course covers how to productionalize machine learning solutions using Kubeflow.

Try this learning path for free
Access this learning path and other top-rated tech content with a free trial.
Have questions? Get them answered now.
What You'll Learn
  • This path teaches the following skills
  • Design and build data processing systems on Google Cloud Platform
  • Lift and shift your existing Hadoop workloads to the Cloud using Cloud Dataproc.
  • Process batch and streaming data by implementing autoscaling data pipelines on Cloud Dataflow
  • Manage your data Pipelines with Data Fusion and Cloud Composer.
  • Derive business insights from extremely large datasets using Google BigQuery
  • Learn how to use pre-built ML APIs on unstructured data and build different kinds of ML models using BigQuery ML.
  • Enable instant insights from streaming data
Prerequisites
  • Participants should have experience with one or more of the following:
  • • A common query language such as SQL
  • • Extracting, Loading, Transforming, cleaning, and validating data
  • • Designing pipelines and architectures for data processing
  • • Integrating analytics and machine learning capabilities into data pipelines
  • • Querying datasets, visualizing query results and creating reports
Related topics
  • BigQuery
  • Dataflow
  • Dataproc
  • ML APIs
  • Data Fusion
  • Bigtable
Not sure where to start?
With over 500 assessments to choose from, you can see where your skills stand and receive adaptive learning recommendations to fill knowledge gaps in as little as 10 minutes.
Learn more

Join our learners and upskill
in leading technologies