Featured resource
pluralsight tech forecast
2025 Tech Forecast

Which technologies will dominate in 2025? And what skills do you need to keep up?

Check it out
Hamburger Icon
  • Course
    • Libraries: If you want this course, consider one of these libraries.
    • Data

Transform Data Using PySpark

Master large-scale data manipulation and analysis with PySpark. This course covers essential techniques for handling data, creating efficient workflows, and using custom functions to streamline complex tasks.

Pinal Dave - Pluralsight course - Transform Data Using PySpark
by Pinal Dave

What you'll learn

Efficient data manipulation is critical for processing large-scale datasets effectively.

In this course, Transform Data Using PySpark, you’ll gain the ability to manipulate, clean, and analyze large datasets using PySpark.

First, you’ll explore how to read and write data using various formats with schema specifications.

Next, you’ll discover how to perform advanced transformations, including grouping, joins, and window functions, as well as handle data cleaning tasks like managing missing, null, and duplicate values.

Finally, you’ll learn how to create custom functions, including UDFs, UDTFs, and vectorized UDFs, to extend PySpark's functionality for specific analytical needs.

When you’re finished with this course, you’ll have the skills and knowledge of PySpark needed to create efficient and reusable workflows for any data-driven challenge.

Table of contents

About the author

Pinal Dave - Pluralsight course - Transform Data Using PySpark
Pinal Dave

Pinal Dave is a Pluralsight Developer Evangelist.

More Courses by Pinal