partial_fit: sgdregression

# How to train SGDRegression on micro-batches via partial_fit.

In this part we're going to train our first `SGDRegressor`. Let's start by simulating a regression dataset that we'll use.

``````import numpy as np
import pandas as pd
from sklearn.datasets import make_regression
from sklearn.linear_model import SGDRegressor, LinearRegression
from sklearn.model_selection import train_test_split

# Prepare Data
X, y, w = make_regression(n_features=2, n_samples=4000,
random_state=42, coef=True, noise=1.0)
y = y + 1.5

X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.5,
random_state=42)
``````

We'll run a baseline `LinearRegression` model first, so we have something to compare against once we train a `SGDRegressor` on a stream of data..

``````# Run a Baseline Model
mod_lm = LinearRegression()
mod_lm.fit(X_train, y_train)

# Keep the MSE number around for safe-keeps.
normal_mse_test = np.mean((mod_lm.predict(X_test) - y_test)**2)
``````

Given our benchmark we can start learning via `._partial_fit()`.

``````# Run for Stats
mod_pac = SGDRegressor()
data = []

for i, x in enumerate(X_train):
# This is where we learn on a single datapoint
mod_pac.partial_fit([x], [y_train[i]])

# This is where we measure and save stats
data.append({
'c0': mod_pac.intercept_[0],
'c1': mod_pac.coef_.flatten()[0],
'c2': mod_pac.coef_.flatten()[1],
'mse_test': np.mean((mod_pac.predict(X_test) - y_test)**2),
'normal_mse_test': normal_mse_test,
'i': i
})

df_stats = pd.DataFrame(data)
``````

These stats can be expected via;

``````import altair as alt

alt.data_transformers.disable_max_rows()

pltr1 = (pd.melt(df_stats[['i', 'c1', 'c2']], id_vars=["i"]))
pltr2 = (pd.melt(df_stats[['i', 'normal_mse_test', 'mse_test']], id_vars=["i"]))

p1 = (alt.Chart(pltr1, title='SGD evolution of weights')
.mark_line()
.encode(x='i', y='value', color='variable', tooltip=['i', 'value', 'variable'])
.properties(width=300, height=150)
.interactive())

p2 = (alt.Chart(pltr2, title='SGD evolution of mse')
.mark_line()
.encode(x='i', y='value', color='variable', tooltip=['i', 'value', 'variable'])
.properties(width=350, height=150)
.interactive())

p1 | p2
``````

If you're unfamiliar with the altair API, you may appreciate our course on it.