scikit meta:
voting
It is very time consuming to write your own scikit-learn models but you can tweak existing ones by applying meta-models on them. These estimators take a model as input and add behavior to it. In this series of video's well highlight a few of them from scikit-learn and scikit-lego.
Notes
To run the code yourself, you'll need to make sure the following tools are loaded.
import numpy as np
import matplotlib.pylab as plt
from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import VotingClassifier
from sklearn.datasets import make_classification
The classification problem that we'll try to solve is generated.
X, y = make_classification(n_samples=2000, n_features=2,
n_redundant=0, random_state=21,
class_sep=1.75, flip_y=0.1)
plt.scatter(X[:, 0], X[:, 1], c=y, s=5);
To get the pretty plots you'll need the following function.
def make_plots():
X_new = np.concatenate([
np.random.uniform(np.min(X[:, 0]), np.max(X[:, 0]), (20000, 1)),
np.random.uniform(np.min(X[:, 1]), np.max(X[:, 1]), (20000, 1))
], axis=1)
plt.figure(figsize=(16, 4))
plt.subplot(141)
plt.scatter(X[:, 0], X[:, 1], c=y, s=5)
plt.title("original data")
plt.subplot(142)
plt.scatter(X_new[:, 0], X_new[:, 1], c=clf1.predict_proba(X_new)[:, 1], s=5)
plt.title("ens1")
plt.subplot(143)
plt.scatter(X_new[:, 0], X_new[:, 1], c=clf2.predict_proba(X_new)[:, 1], s=5)
plt.title("ens2")
plt.subplot(144)
plt.scatter(X_new[:, 0], X_new[:, 1], c=clf3.predict_proba(X_new)[:, 1], s=5)
plt.title("ens3");
To train the three models and get the visuals you can run:
clf1 = LogisticRegression().fit(X, y)
clf2 = KNeighborsClassifier(n_neighbors=10).fit(X, y)
clf3 = VotingClassifier(estimators=[('clf1', clf1), ('clf2', clf2)],
voting='soft',
weights=[10.5, 2.5])
clf3.fit(X, y)
make_plots()
Make sure you play with the weights
parameter to get a feel of the effect.
Feedback? See an issue? Something unclear? Feel free to mention it here.
If you want to be kept up to date, consider signing up for the newsletter.