Calmcode - decorators: usage


1 2 3 4 5 6 7 8 9 10

Decorators are extremely powerful. A great use-case for them is to add logging to pandas pipelines. If you're interested in more information on the pandas example below, make sure you watch this course.

The pandas-logger decorator is defined below.

from functools import wraps
import datetime as dt

def log_step(func):
    def wrapper(*args, **kwargs):
        tic =
        result = func(*args, **kwargs)
        time_taken = str( - tic)
        print(f"just ran step {func.__name__} shape={result.shape} took {time_taken}s")
        return result
    return wrapper

And you can see it being applied here;

import pandas as pd

df = pd.read_csv('')

def start_pipeline(dataf):
    return dataf.copy()

def set_dtypes(dataf):
    return (dataf
            .assign(date=lambda d: pd.to_datetime(d['date']))
            .sort_values(['currency_code', 'date']))

def remove_outliers(dataf, min_row_country=32):
    countries = (dataf
                .agg(n=('name', 'count'))
                .loc[lambda d: d['n'] >= min_row_country]
    return (dataf
            .loc[lambda d: d['currency_code'].isin(countries)])

df_new = (df
  .pipe(remove_outliers, min_row_country=20))

Another example of a useful decorator can be found in the retry package.

from retry import retry

import logging

@retry(ValueError, tries=5, delay=0.5)
def randomly_fails(p=0.5):
    if random.random() < p:
        raise ValueError("no bueno!")
    return "Done!"


There's plenty of other useful decorators. There's the lru-cache but there's also the multifile decorator that we mention in the video.