Improving Azure Data Lake Performance
Running queries in Azure Data Lake? Are your queries costing too much? This course will help you learn how to take control of your Data Lake. Grab those pesky queries by the scruff of the neck and improve Azure Data Lake performance!
What you'll learn
OK, so you are using Azure Data Lakes, and you think it's great. You just wish you could improve the performance of your U-SQL queries. Why does that query always read your entire data set? Why does this query take forever to complete? Like anything else in the Big Data world, your Azure Data Lake has to be structured around your data. This course, Improving Azure Data Lake Performance, will show you how to put the right structure in place. Then watch the magic start to happen! First, you'll see how an Azure Data Lake works behind the scenes – how it handles different types of data and how the storage of that data can be optimized. Next, you'll see how it's possible to optimize non-structured data. Finally, you'll be shown how structuring your data opens up a world of possibilities, including horizontal and vertical partitioning. This is where the real power of the Azure Data Lake comes to light! Horizontal partitioning allows you to defer a lot of control to the Data Lake, whereas vertical partitioning allows you – the developer – to take total control of how your data is partitioned and distributed within the Data Lake. When you're finished with this course, you'll understand how you can better optimize your jobs and save some cash. Software required: Visual Studio Community Edition 2017 with the Azure Data Lake and Stream Analytics Tools installed.
Table of contents
- Version Check 0m
- Introduction 2m
- Why Organize a Data Lake? 4m
- How Data Is Stored and Processed in a Data Lake 2m
- Cost vs. Speed 4m
- Determining the Correct Number Of ADLAUs 8m
- Comparing Job Costs 3m
- How to Organize Files in a Data Lake 2m
- Azure Data Lake Databases 5m
- Azure Data Lake Problems 5m
- Indexes 5m
- Partition Schemes 3m
- Summary 1m
- Introduction 1m
- What's so Great About Vertical Partitioning? 2m
- Creating a Vertically Partitioned Table 3m
- Inserting Data into a Table with No Partitions 1m
- Creating Partitions 2m
- Inserting with Integrity 3m
- Querying Partitions 2m
- Managing Partitions 4m
- Using Multiple Keys in Vertical Partitioning 5m
- Summary 1m