Explore Python Libraries: Speed Up HTTP Tests with VCR.py

Jul 31, 2020 • 5 Minute Read

Introduction

In the real world, writing unit and integration tests that include HTTP requests can be fraught with peril. Slow and indeterministic tests can be the norm. How can you write deterministic, repeatable tests for HTTP requests within your Python app or service? How can you run these same tests offline?

VCR.py is the answer.

The VCR.py library records the responses from HTTP requests made within your unit tests. The first time you run your tests using VCR.py is like any previous run. But the after VCR.py has had the chance to run once and record, all subsequent tests are:

Fast! No more waiting for slow HTTP requests and responses in your tests.
Deterministic. Every test is repeatable since they run off of previously recorded responses.
Offline-capable! Every test can now run offline.

In the following sections, you will learn how to get started using VCR.py and how you can use its API to achieve the benefits mentioned above.

Installation

VCR.py requires Python version 3.5 or greater and is available to download using pip.

To install VCR.py using pip, you can run the following command:

      pip install vcrpy

Once the library is downloaded, you can import it and start creating recordings that VCR.py calls "cassettes". VCR.py works primarily via the @vcr decorator. You can import this decorator by writing: import vcr.

Writing HTTP Unit Tests

With the library installed and imported, you can now begin capturing HTTP requests within your unit tests. Every time VCR.py captures a unit test, it creates a cassette. A cassette is a .yaml file that VCR.py uses to record the response metadata from an HTTP request. You can create a basic cassette like this:

      import vcr
import requests
import unittest

class GoogleTest(unittest.TestCase):

    @vcr.use_cassette('fixtures/cassettes/google_get.yaml')
    def test_get_google_home_page(self):
        response = requests.get('https://google.com')
        self.assertEqual(response.status_code, 200)
    

After you have installed the "requests" library, you can execute the above test using unittest via the following command:

      python -m unittest ./path/to/unit/test/file.py

The first time that the above test was run on my machine, it finished in ~0.5 seconds and the google_get.yaml cassette was created within the fixtures directory. However, the second time this test was run, it finished in 0.03 seconds - 16x as fast as the original. This, however, is a contrived example. The Google homepage is an extremely fast GET request from the start. But I am sure that you can see how VCR.py makes unit tests much faster. The primary speed benefit is realized when your unit tests contain many HTTP requests that are likely much slower than the preceding example.

Record Modes

VCR.py has four recording modes. These include:

once
new_episodes
none
all

These recording modes tell VCR.py how to record its cassettes, if at all. The first recording mode, once, is the default mode. In this mode, the first time that a unit test is run, VCR.py will record the HTTP requests within it and create a cassette. Every time after, VCR.py will verify the test against the created cassette, creating a new cassette if none exists. In this mode, an error will also be thrown if VCR.py captures an HTTP request that is not verifiable against an existing cassette.

The second mode, new_episodes, builds off of the first mode in that it will stop any error from being thrown when new HTTP requests are detected. Instead, this mode will simply record the new request(s) and update the existing cassette.

The none recording mode is exactly like the first, except that it requires there to be an existing cassette in place. This record mode, like once, will ensure that new requests are not made.

The all recording mode basically tells VCR.py to always create a new cassette. By using this recording mode, you will lose the majority of the benefits of VCR.py, so use it with caution! The primary benefit of this recording mode is temporary. It is useful when you want to force VCR.py to rebuild a particular cassette.

You can set the recording mode of your unit test by using the record_mode named argument to the @vcr decorator like this:

      @vcr.use_cassette('fixtures/cassettes/google_get.yaml', record_mode='all')

Conclusion

In this guide, you learned about the VCR.py library in-depth. You learned how you can use VCR.py to make your Python tests repeatable, offline-capable, and fast! You learned about the different record modes that VCR.py offers and how you can use them to write unit tests. For more information, including advanced usage, check out the VCR.py documentation.