-
Course
- Data
Scraping Your First Web Page with Python
This course covers the important tools for retrieving web content using HTTP libraries such as Requests, Httplib2 and Urllib, as well as powerful technologies for web parsing. These include Beautiful Soup, which is a popular library, and Scrapy, which is a powerful, production-grade framework.
What you'll learn
Web scraping is an important technique that is widely used as the first step in many workflows in data mining, information retrieval, and text-based machine learning. In this course, Scraping your First Web Page with Python, you will gain the ability to apply different scraping techniques including Beautiful Soup, and Scrapy. First, you will learn and use various HTTP client libraries such as Requests, httplib2, and urllib to download HTML content. Next, you will discover how Beautiful Soup is an extremely popular Python library that does better than regex in important ways. You will see how Beautiful Soup fixes up badly formed HTML, and constructs a nice parse tree that can be traversed and queried. Finally, you will add to your toolkit the knowledge of Scrapy, which is a full-fledged web scraping framework that combines the steps of retrieving and parsing web content and does so at production-scale. When you’re finished with this course, you will have the skills and knowledge to identify the relative strengths and use-cases of different web retrieval and scraping technologies such as regular expressions, Beautiful Soup, and Scrapy.
Table of contents
- Version Check | 16s
- Module Overview | 1m 8s
- Prerequisites and Course Outline | 1m 21s
- HTTP Requests and Responses | 5m 45s
- Web Scraping | 2m 24s
- HTTP Client Libraries | 4m 21s
- Making GET Requests Using httplib2 | 7m 18s
- Making OPTIONS, POST, PUT Requests with httplib2 | 4m 7s
- Handling Redirects with httplib2 | 3m 33s
- Making HTTP Requests and Parsing URLs Using urllib | 7m 29s
- GET and POST Requests Using the Requests Library | 4m 36s
- Handling Redirects with the Requests Library | 3m 16s
- Module Summary | 1m 17s
About the author
A problem solver at heart, Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework.
More Courses by Janani