Introduction

We're starting an open source project on behalf of this website! In this series of video's well set up the project locally, turn our python code into a python project and also make the first version of it pip-installable.

The goal of this project (named Clumper) will be to offer tools that make it easier to deal with long lists of dictionaries/json-like objects. The API will look something like this;

from clumper import Clumper

list_of_dicts = [
  {'date': '2014-01-01', 'values': [1, 2, 3]},
  {'date': '2014-01-02', 'values': [1, 2, 3, 4]},
  ...
]

(Clumper(list_of_dicts)
  .keep(lambda d: d['date'] == '2014-01-01')
  .mutate(sum_values=lambda d: sum(d['values'])))

This code will take our list of dictionaries, grab all the instances where the date is equal to 2014-01-01 and for those instances add a key called sum_values to is equal to the sum of the values list in each of those dictionaries. The code that might do this looks like this:

class Clumper:
    def __init__(self, blob):
        self.blob = blob

    def keep(self, *funcs):
        data = self.blob
        for func in funcs:
            data = [d for d in data if func(d)]
        return Clumper(data)

    def head(self, n):
        return Clumper([self.blob[i] for i in range(n)])

    def tail(self, n):
        return Clumper([self.blob[-i] for i in range(1, n+1)])

    def select(self, *keys):
        return Clumper([{k: d[k] for k in keys} for d in self.blob])

    def mutate(self, **kwargs):
        data = self.blob
        for key, func in kwargs.items():
            for i in range(len(data)):
                data[i][key] = func(data[i])
        return Clumper(data)

    def sort(self, key, reverse=False):
        return Clumper(sorted(self.blob, key=key, reverse=reverse))

Note that this is the code that we start with and it is the exact same code as what we ended with in our videos on method chains. We'll add more and more features to this class but in this series of videos we'll focus on getting a project set up where we might work in.