logo

... logging.


Print statements can only do so much. You also need to log to files if you want to debug something that is in production. That is why in this series of videos we'll show you how to set up logging for your project and to also show you how it works.


Notes

This time we've made a big change to our logger.py file.

import logging

logger = logging.getLogger(__name__)

# the handler determines where the logs go: stdout/file
shell_handler = logging.StreamHandler()
file_handler = logging.FileHandler("debug.log")

logger.setLevel(logging.DEBUG)
shell_handler.setLevel(logging.WARNING)
file_handler.setLevel(logging.DEBUG)

# the formatter determines what our logs will look like
fmt_shell = '%(levelname)s %(asctime)s %(message)s'
fmt_file = '%(levelname)s %(asctime)s [%(filename)s:%(funcName)s:%(lineno)d] %(message)s'

shell_formatter = logging.Formatter(fmt_shell)
file_formatter = logging.Formatter(fmt_file)

# here we hook everything together
shell_handler.setFormatter(shell_formatter)
file_handler.setFormatter(file_formatter)

logger.addHandler(shell_handler)
logger.addHandler(file_handler)

We've also added more logging statements to the other files.

job.py

import sys 

from logger import logger
from summarise import summary

if __name__ == "__main__":
    logger.debug("Program started.")
    try:
        ticker = sys.argv[1]
        logger.info(f"Will find summary for {ticker}.")
        print(f"The average stock price is {summary(ticker)}")
        logger.debug("Program ended with success.")
    except BaseException:
        logger.error("Error happened!", exc_info=True)

summerise.py

from fetch import download_data
from logger import logger 

def summary(ticker):
    logger.info(f"About to download data.")
    dataf = download_data()
    logger.debug(f"Dataset downloaded. shape={dataf.shape}")
    logger.debug(f"Dataset downloaded. columns={dataf.columns}")
    return dataf[ticker].mean()

fetch.py

import pandas as pd

from logger import logger 

def download_data():
    url = 'https://calmcode.io/datasets/stocks.csv'
    logger.debug(f"Fetching from {url}.")
    df = pd.read_csv(url)
    logger.info(f"File downloaded from {url}.")
    return df

Feedback? See an issue? Something unclear? Feel free to mention it here.

If you want to be kept up to date, consider getting the newsletter.