Forward Fill

September 2026 · 9 min read

Forward fill copies the most recent observed value into a later gap. It is also called last observation carried forward. The operation is simple; the useful part is deciding when that earlier value still describes what you need.

A setting recorded whenever it changes can remain valid between updates. A missing temperature measurement may have changed since the previous reading. In both cases the code can carry a value forward, but only the first has a “persists until changed” interpretation—and only if the change log is complete.

Remember the last observation, not the last filled cell

Scan one entity’s records in time order. On an observed value, remember its value and timestamp. On a gap, consider that stored observation. Before the first observation there is nothing to carry, so leading gaps remain missing.

Our example has irregular time gaps. A reading of 3 arrives at minute 1; the next real reading, 5, arrives at minute 13. Unlimited forward fill would repeat 3 at minutes 2, 4, and 12. An age limit can stop that last, stale fill.

At a three-minute limit, minute 4 is still eligible because its source is exactly three minutes old. Minute 12 is not. Filling minute 4 must not reset the observation’s timestamp to 4: that would allow a chain of copies to make old evidence appear fresh.

Keep entities, order, and gap length explicit

Sort by entity and time before filling, and fill within each entity. A device with no earlier observation must not inherit the previous device’s final reading. If multiple records share a timestamp, resolve their order or combine them according to the data contract; a sort alone does not establish which one is authoritative.

Pandas’ ffill(limit=2) limits consecutive missing rows filled after an observation. It does not mean two minutes. On irregular data, a single missing row could be hours after its source. Track the original observation time for an elapsed-time limit.

import numpy as np
import pandas as pd

df = pd.DataFrame({
    "device": ["A"] * 7 + ["B"] * 3,
    "minute": [0, 1, 2, 4, 12, 13, 14, 2, 3, 4],
    "value": [np.nan, 3, np.nan, np.nan, np.nan, 5, np.nan,
              np.nan, 9, np.nan],
}).sort_values(["device", "minute"])
assert not df.duplicated(["device", "minute"]).any()

was_missing = df["value"].isna()
observed_at = df["minute"].where(~was_missing)
last_observed_at = observed_at.groupby(df["device"]).ffill()
df["age"] = df["minute"] - last_observed_at
candidate = df.groupby("device")["value"].ffill()
df["filled"] = candidate.where(df["age"] <= 3)
df["was_missing"] = was_missing

assert np.allclose(df.loc[df.device == "A", "filled"],
                   [np.nan, 3, 3, 3, np.nan, 5, 5], equal_nan=True)
assert np.allclose(df.loc[df.device == "B", "filled"],
                   [np.nan, 9, 9], equal_nan=True)
print(df.loc[df.device == "A", ["minute", "value", "age", "filled"]])

Device B’s first row stays missing even though device A has observations. Device A’s filled sequence is [missing, 3, 3, 3, missing, 5, 5]. The code’s age column also makes staleness inspectable instead of silently hiding it inside a replacement.

“Earlier timestamp” must also mean “available then”

Forward fill cannot by itself guarantee a valid historical prediction. A measurement may describe an earlier event but arrive later. A backtest built from the final, corrected dataset can accidentally use information that the live system did not yet have.

Historical features need the appropriate event-time and availability rules. A backward as-of join can choose the latest eligible observation within an age tolerance. It still depends on choosing the right time key and excluding records unavailable at the decision time.

Carry valid history across a time split

If production retains a device’s earlier history, the first validation prediction may use its last available training-period observation. Crossing that chronological boundary is not inherently leakage; using information unavailable at the prediction cutoff is. Keep the age limit in force across the boundary.

A rolling backtest may incorporate new observations as they arrive. A batch forecast made once for an entire future horizon cannot use observations collected partway through that horizon. Reproduce the actual update schedule, as described in time-series cross-validation.

Forward filling estimates no global mean, but it maintains per-entity state. That state and its timestamps must be initialized and updated consistently during evaluation and serving. A shuffled split usually destroys the interpretation of “the history available before this prediction.”

Use the carry as a feature with an age, not a recovered truth

For a complete change log, carrying a setting forward can recover the state between changes. For a changing physical measurement, it is a persistence assumption. Assess errors during long outages, rapid changes, and periods when the recording process differs.

Leading gaps need an explicit policy: leave them missing for a capable estimator, use a training-derived baseline with an indicator, or abstain when the task requires a recent measurement. A backward fill obtains a later observation and is not an automatic repair for an online feature.

Repeated copies are not independent new measurements. They can flatten apparent variation and distort correlations if analyzed as observed values. Keep the raw record and use the carried representation for its stated purpose.

C++: one entity with a maximum observation age

The following pass assumes the same immediate-availability convention as the figure. It requires nonnegative, strictly increasing minute timestamps and retains the timestamp of the last real observation. Call it separately for each entity. The scan is linear in the number of rows; it uses constant state plus the returned output vector.

#include <cmath>
#include <iostream>
#include <limits>
#include <stdexcept>
#include <vector>

struct Sample { int minute; double value; };

std::vector<double> forwardFill(const std::vector<Sample>& samples, int maxAge) {
    if (maxAge < 0) throw std::invalid_argument("negative age limit");
    const double NA = std::numeric_limits<double>::quiet_NaN();
    double last = NA;
    int lastObservedAt = 0, previousTime = -1;
    std::vector<double> result;
    for (const auto& sample : samples) {
        if (sample.minute < 0 || sample.minute <= previousTime)
            throw std::invalid_argument("nonnegative, strictly increasing times required");
        if (std::isinf(sample.value)) throw std::invalid_argument("infinite value");
        previousTime = sample.minute;
        if (!std::isnan(sample.value)) {
            last = sample.value;
            lastObservedAt = sample.minute;
            result.push_back(sample.value);
        } else if (!std::isnan(last) && sample.minute - lastObservedAt <= maxAge) {
            result.push_back(last);  // A fill does not update lastObservedAt.
        } else {
            result.push_back(NA);
        }
    }
    return result;
}

int main() {
    const double NA = std::numeric_limits<double>::quiet_NaN();
    const std::vector<Sample> samples = {
        {0,NA}, {1,3}, {2,NA}, {4,NA}, {12,NA}, {13,5}, {14,NA}
    };
    const auto result = forwardFill(samples, 3);
    for (std::size_t i = 0; i < result.size(); ++i) {
        if (i) std::cout << " ";
        if (std::isnan(result[i])) std::cout << "NA";
        else std::cout << result[i];
    }
    std::cout << "\n";
}

NA 3 3 3 NA 5 5

References and further reading

• pandas: forward fill — direction and the consecutive-row limit.
• pandas: grouped forward fill — propagating values within groups.
• pandas: as-of joins — directional time matching, grouping, and tolerances.
• Feast: point-in-time joins — matching feature history to each example’s timestamp.