retry.py

Let's pretend that we have a function that makes a request to an endpoint.

import random
from requests.exceptions import HTTPError

def pretend_request():
    if random.random() < 0.01:
        raise HTTPError("Something went wrong!")
    return {"status": "alive"}

This function is fake, but it similates something that tends to happen all the time. It simulates a HTTPError one percent of the time.

The error won't happen that often, but it will happen once in a while. If you're running a batch script that communicates with many services you may get unlucky and the job could break.

Just try running this for-loop.

# You'll likely hit an error
for i in range(500):
    pretend_request()

Enter retry

This is where the retry library might help. It's a library with a single decorator that can tell the function to try again after failing. Here's how you can use it:

from retry import retry

@retry(tries=3, delay=0.1)
def pretend_request():
    if random.random() < 0.01:
        raise HTTPError("Something went wrong!")
    return {"status": "alive"}

With these settings, the retry function will retry three times and between each retry it will wait for 0.1 seconds. If you were now to run the for-loop, you shouldn't see any errors anymore.

# You'll likely hit an error
for i in range(500):
    pretend_request()

Logging

The @retry decorator will also appear in your logs.

import random
import logging
from requests.exceptions import HTTPError
from retry import retry

logging.basicConfig()

@retry(HTTPError, tries=10, delay=0.1, backoff=2)
def pretend_request():
    if random.random() < 0.1:
        raise rq.exceptions.HTTPError("Something went wrong!")
    return {"status": "alive"}

for i in range(100):
    pretend_request()

If you pay close attention to the logs, you can see that the backoff setting will double the delay time on every hit. That's why you might see a few logs that look like:

WARNING:retry.api:Something went wrong!, retrying in 0.1 seconds...
WARNING:retry.api:Something went wrong!, retrying in 0.2 seconds...
WARNING:retry.api:Something went wrong!, retrying in 0.4 seconds...

Configuration

You could also configure the @retry decorator to be more specific.

import random
from retry import retry
from requests.exceptions import HTTPError


@retry(HTTPError, tries=3, delay=0.1)
def pretend_request():
    if random.random() < 0.01:
        raise HTTPError("Something went wrong!")
    return {"status": "alive"}

Now, the function will only retry if detects a HTTPError. If you were to configure it to only respond to ValueError and TypeError then any other errors will be ignored.

@retry((ValueError, TypeError), tries=3, delay=0.1)
def pretend_request():
    if random.random() < 0.01:
        raise HTTPError("Something went wrong!")
    return {"status": "alive"}

# You'll likely hit an error again
for i in range(500):
    pretend_request()