Squared Error: Why the Best Constant Is the Mean

September 2026 · 5 min read + loss explorer

Squared error asks a constant prediction to balance the sizes of its overpredictions and underpredictions. That balance produces the mean, not merely a convenient default.

The loss and its prediction scale

The raw score $F$ is the reported prediction. For one observed target $y$, use half squared error:

$$\ell(y,F)=\frac12(y-F)^2.$$

The factor $1/2$ simplifies derivatives. Ordinary squared error and mean squared error have the same best constant because multiplying the whole objective by a positive number does not change where it is minimized.

The score derivative is $\partial\ell/\partial F=F-y$, so the negative-gradient correction target is $y-F$. Underprediction requests an increase; overprediction requests a decrease. Larger errors request larger corrections.

Derive the best constant

Every row receives the same prediction $c$. Keep the observed $y_i$ fixed and add their losses:

$$J(c)=\frac12\sum_{i=1}^n(y_i-c)^2.$$

For each term, the chain rule gives $\tfrac12\cdot2(y_i-c)\cdot(-1)=c-y_i$. Summing those derivatives gives

$$\begin{aligned}J'(c)&=\sum_i(c-y_i)\\&=nc-\sum_i y_i.\end{aligned}$$

At the optimum the aggregate slope is zero:

$$nc^*=\sum_i y_i\quad\Longrightarrow\quad c^*=\frac1n\sum_i y_i=\bar y.$$

Why is this a minimum rather than a maximum? The second derivative is $J''(c)=n>0$. The objective curves upward everywhere, so this is its unique global minimum.

Read the balance in four observations

For targets $[1,2,3,10]$, the mean is $c^*=16/4=4$. Their residuals are $[-3,-2,-1,6]$. The first three rows together request $-6$; the last requests $+6$. No net first-order correction remains.

The total half squared loss is $(9+4+1+36)/2=25$. Predicting 3 instead gives $(4+1+0+49)/2=27$. The distant target 10 pulls the optimum above the three small targets because its error grows quadratically.

Total training loss against a shared constant score, with the minimizing score or interval marked. The targets and numerical optimum are stated in the text. — The curve sums the stated loss over all displayed observations. The orange mark shows the minimizing score or interval; the blue point follows your chosen constant. Open the computed figure.

What changes with observation weights?

With fixed weights $w_i\ge0$ and positive total weight, minimize $J(c)=\tfrac12\sum_i w_i(y_i-c)^2$. The same derivative calculation gives

$$c^*=\frac{\sum_i w_i y_i}{\sum_i w_i}.$$

This is the weighted mean. It follows from the objective, rather than from replacing the targets with weighted target values and taking an ordinary average.

What this tells you about a fitted model

A model restricted to one constant estimates the training mean. With a flexible function of $x$, minimizing expected squared loss targets the conditional mean $\mathbb E[Y\mid x]$, when the relevant moments exist. Large target errors can dominate the fit; compare absolute error and Huber loss when that influence is undesirable.

Sources and reproducible figures

Scikit-learn: squared error and the mean

The examples and plots are computed by this reproduction script, which also checks the stated minima. Inspect the figure data.