Splines and GAMs

September 2026 · 12 min read

Suppose electricity use rises on both very cold and very hot days. A straight line cannot represent that shape; a high-degree polynomial can bend wildly near the ends. A spline fits low-degree polynomial pieces that join at locations called knots. A generalized additive model (GAM) combines several smooth effects, such as temperature and time of day, while keeping each effect inspectable.

This article builds one spline by hand, fits the same nine observations at three smoothing strengths, and then turns smooth effects into a GAM prediction. Familiarity with basis regression and GLM link functions is enough.

One Smooth Curve from Joined Pieces

A cubic spline uses polynomials of degree at most three. At a simple interior knot, the pieces agree in value, slope, and second derivative. The third derivative may change: a spline can change its bending behavior without a jump or a sharp corner.

Here is a complete example with one knot at $x=2$:

For $x\le2$, the extra term is zero and $f(x)=1+x$. For $x>2$, a cubic correction gradually bends the line downward. It does not switch on with a jump.

With $K$ interior knots, a general cubic spline can be written in the truncated power basis:

The input-to-basis transformation is nonlinear, but the model is still linear in its coefficients. Ordinary or penalized least squares can estimate those coefficients.

B-splines represent the same spline space using better-scaled basis functions with local support: each one is nonzero only on a few neighboring knot intervals. Changing one coefficient therefore has a local effect. Refitting after changing an observation can still change many coefficients and the curve globally. Local basis support is not a guarantee of local influence.

A natural cubic spline adds boundary conditions that yield linear tails when extended beyond the boundary knots. This avoids unrestricted cubic tails, but a linear extrapolation is still an assumption, not evidence about unobserved inputs.

Why Smoothing Is Different from Knot Placement

Imagine putting a knot near every observation. The model can now follow each small fluctuation. A curvature penalty makes it pay for doing so:

The first term measures training error. The integral measures squared curvature: a straight line has zero penalty. For this ordinary second-derivative penalty, small $\lambda$ allows more bending and large $\lambda$ suppresses it. As $\lambda\to\infty$, the solution approaches a least-squares line. Extra penalties on the constant and linear parts would change that limit.

Two related constructions are easy to confuse:

• A regression spline uses a chosen finite basis. With no penalty, it fits the best curve available in that basis; it need not interpolate the observations.
• A cubic smoothing spline solves the curvature-penalized problem over a larger function space. Its solution is a natural cubic spline with knots at the distinct observed inputs. With distinct inputs, the $\lambda\downarrow0$ limit interpolates them.

For a finite basis $f(x)=\sum_j\theta_j B_j(x)$, let $B_{ij}=B_j(x_i)$ and $\Omega_{jk}=\int B_j''(t)B_k''(t)\,dt$. Then smoothing is a linear-system problem:

Solve this system rather than forming an explicit inverse. Numerical conditioning and identifiability still matter. The mgcv smooth-term documentation describes practical basis and penalty choices, including cubic, cyclic, and tensor-product smooths.

Nine Observations, Three Smoothing Strengths

Use $x=(0,1,\ldots,8)$ and $y=(0.2,1.4,0.6,2.7,1.6,3.2,1.4,1.9,0.5)$. These deliberately noisy toy observations make the tradeoff visible. All three curves below solve the same cubic smoothing-spline problem; only $\lambda$ changes.

For a fixed $\lambda$ and fixed inputs, the Gaussian fitted values are linear in the response: $\hat y=S_\lambda y$. The effective degrees of freedom (EDF) are $\operatorname{tr}(S_\lambda)$, which measures how responsive fitted values are to changes in the observed responses.

EDF can be fractional because smoothing shrinks directions continuously. The fixed-$\lambda$ trace does not count the additional adaptivity of selecting $\lambda$ from $y$. A centered GAM smooth also usually excludes the intercept from its reported EDF, so compare like with like.

import numpy as np
from scipy.interpolate import make_smoothing_spline

x = np.arange(9, dtype=float)
y = np.array([0.2, 1.4, 0.6, 2.7, 1.6, 3.2, 1.4, 1.9, 0.5])
for lam in (0.0, 0.5, 100.0):
    fit = make_smoothing_spline(x, y, lam=lam)
    # A linear smoother applied to each unit vector reveals its columns.
    S = np.column_stack([
        make_smoothing_spline(x, e, lam=lam)(x)
        for e in np.eye(len(x))
    ])
    print(lam, np.trace(S), np.sum((y - fit(x)) ** 2))

From One Smooth to a GAM Prediction

A GAM combines smooth effects through a GLM link function:

For Gaussian responses with an identity link, the right side is the predicted mean. For a binary response with a logit link, it is log-odds. Each smooth has its own basis and usually its own smoothing parameter.

Consider a fitted churn model with intercept $-1.5$, a tenure contribution of $+0.8$, and a usage contribution of $-0.4$ for one customer. Add contributions before applying the inverse link:

The tenure contribution is not an 80-percentage-point increase in churn. It is $0.8$ on the log-odds scale, relative to the term's centering convention. Comparing two tenure values uses $f_{\rm tenure}(a)-f_{\rm tenure}(b)$; its exponential is a conditional odds ratio.

Without a constraint, a constant could move between a smooth and the intercept without changing any prediction. Most smooths therefore satisfy $\sum_i f_j(x_{ij})=0$. This makes term plots interpretable relative to a common baseline; it does not force the smooth to be zero at the mean input. See mgcv's identifiability constraints.

Fit, Check, and Interpret in That Order

1. Choose the response family and validation split. Counts, binary outcomes, and continuous measurements need different observation models. Preserve groups or time order when deployment requires it.
2. Give the basis enough room. Start with a plausible basis size and estimate smoothing with REML, GCV, or inner validation. A strong penalty can suppress excess flexibility; a small basis cannot represent a shape it never contained.
3. Check the basis after fitting. Inspect residual patterns and refit with a larger basis if needed. EDF near the available maximum is a useful warning, but lower EDF alone does not prove the basis sufficient. The mgcv basis-dimension guide gives concrete checks.
4. Evaluate the complete pipeline on held-out data. Learn data-dependent knots, transformations, and smoothing choices on training rows. Compare predictive accuracy and calibration against a simpler linear model.
5. Read effects where there are observations. Plot the input distribution alongside each effect. Sparse regions and extrapolation need particular care; flexible curves cannot create missing evidence.

Concurvity occurs when one smooth can be approximated by other model terms. Predictions may remain useful while individual effect curves become unstable. A term plot is a conditional model summary, and is not automatically a causal effect.

Finally, check what an uncertainty band covers. A pointwise band at each input does not promise simultaneous coverage over the whole curve. Standard bands may condition on estimated smoothing parameters; mgcv's prediction documentation explains its smoothing-parameter uncertainty correction. A band for a mean effect is also different from a prediction interval for a new noisy observation.

Quick Checks