Neel Mishra
← All Posts
From Scratch · C++ » Project Structure

Build Systems

From Scratch Series · 20 min read

Why Build Systems Exist

In the first article of this series, we saw that each .cpp file is compiled independently into an object file, and the linker combines them into an executable. For a trivial project with two files, you can type the compiler command by hand. For a real project with hundreds of source files, dozens of libraries, platform-specific flags, and test suites, manual compilation is impossible. You need a build system.

A build system solves three essential problems:

Without a build system, you either recompile everything on every change (wasting time) or manually track which files to recompile (introducing bugs when you miss one). Build systems automate this entirely.

Makefiles: The Foundation

Make is the oldest and most widely used build tool for C and C++ projects. It was created in 1976 at Bell Labs and remains the underlying build executor on most Unix systems. A Makefile consists of rules, each specifying a target, its prerequisites, and the commands to build it:

# Basic Makefile
CXX      = g++
CXXFLAGS = -std=c++20 -O2 -Wall

# Rule: target: prerequisites
#          command
app: main.o engine.o physics.o
	$(CXX) $(CXXFLAGS) $^ -o $@

main.o: main.cpp engine.h physics.h
	$(CXX) $(CXXFLAGS) -c main.cpp -o main.o

engine.o: engine.cpp engine.h
	$(CXX) $(CXXFLAGS) -c engine.cpp -o engine.o

physics.o: physics.cpp physics.h engine.h
	$(CXX) $(CXXFLAGS) -c physics.cpp -o physics.o

clean:
	rm -f *.o app

Make works by comparing timestamps. When you run make, it checks whether each target is older than its prerequisites. If engine.h was modified after engine.o was built, Make knows to recompile engine.cpp. If main.cpp has not changed and none of its headers have changed, main.o is up to date and is skipped.

Automatic Variables and Pattern Rules

# Automatic variables:
# $@  = the target
# $^  = all prerequisites
# $<  = the first prerequisite

# Pattern rule: any .o can be built from its .cpp
%.o: %.cpp
	$(CXX) $(CXXFLAGS) -c $< -o $@

# Now we only need to specify dependencies:
main.o: main.cpp engine.h physics.h
engine.o: engine.cpp engine.h
physics.o: physics.cpp physics.h engine.h

Automatic Dependency Generation

Manually listing header dependencies is error-prone. GCC and Clang can generate dependency files automatically:

# The -MMD flag generates .d files alongside .o files
CXXFLAGS += -MMD -MP
SRCS = main.cpp engine.cpp physics.cpp
OBJS = $(SRCS:.cpp=.o)
DEPS = $(SRCS:.cpp=.d)

app: $(OBJS)
	$(CXX) $(CXXFLAGS) $^ -o $@

%.o: %.cpp
	$(CXX) $(CXXFLAGS) -c $< -o $@

-include $(DEPS)   # include generated dependency files (- suppresses errors if they don't exist yet)

The .d files contain Make-format dependency rules generated by scanning the #include directives in each source file. This gives you automatic, accurate header dependency tracking.

Limitations of Make

These limitations led to the creation of meta-build systems that generate Makefiles (or Ninja files, or Visual Studio projects) from a higher-level description.

CMake Fundamentals

CMake is the de facto standard meta-build system for C++. It reads a CMakeLists.txt file, understands your project structure, and generates native build files for your platform (Makefiles on Linux, Ninja files, Visual Studio projects on Windows, Xcode projects on macOS).

# CMakeLists.txt - minimal example
cmake_minimum_required(VERSION 3.20)
project(MyApp LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

add_executable(app
    main.cpp
    engine.cpp
    physics.cpp
)

To build:

# Configure (generates build files):
cmake -S . -B build -G Ninja    # or -G "Unix Makefiles" or -G "Visual Studio 17 2022"

# Build:
cmake --build build

# Install:
cmake --install build --prefix /usr/local

CMake handles compiler detection, flag management, header dependency scanning, and cross-platform differences automatically. You describe what to build; CMake figures out how.

Targets: The Core Abstraction

In modern CMake (3.x+), everything revolves around targets. A target is a build artifact (executable, library) with associated properties (source files, compile flags, include paths, dependencies).

# Create a static library target
add_library(engine STATIC engine.cpp physics.cpp)
target_include_directories(engine PUBLIC include/)
target_compile_features(engine PUBLIC cxx_std_20)

# Create an executable that depends on the library
add_executable(app main.cpp)
target_link_libraries(app PRIVATE engine)
# This propagates include dirs and compile features from engine to app

The key commands are:

PUBLIC, PRIVATE, INTERFACE

These keywords control how properties propagate through the dependency graph:

# Header-only library: no source files, only propagates include dirs
add_library(json_lib INTERFACE)
target_include_directories(json_lib INTERFACE include/json/)

# Consumer:
target_link_libraries(app PRIVATE json_lib)
# app now has json_lib's include dirs in its search path
Key insight: Modern CMake is target-centric. Set properties on targets, not global variables. Avoid include_directories() (global); use target_include_directories() (per-target). Avoid add_definitions(); use target_compile_definitions(). The target-based approach ensures properties propagate correctly and only where needed.

Finding External Libraries

CMake has extensive support for finding installed libraries:

# Find an installed library (uses FindXxx.cmake or XxxConfig.cmake):
find_package(Boost 1.80 REQUIRED COMPONENTS filesystem system)
find_package(OpenSSL REQUIRED)
find_package(Threads REQUIRED)

add_executable(app main.cpp)
target_link_libraries(app PRIVATE
    Boost::filesystem
    Boost::system
    OpenSSL::SSL
    Threads::Threads
)

find_package searches for config files (installed by the library) or find modules (shipped with CMake). Modern libraries ship CMake config files (FooConfig.cmake) that define imported targets (like Boost::filesystem). These targets carry all the needed include paths, link flags, and dependencies.

Multi-Directory Projects

# Project structure:
# /CMakeLists.txt       (top-level)
# /src/CMakeLists.txt   (source code)
# /lib/CMakeLists.txt   (internal libraries)
# /tests/CMakeLists.txt (test suite)

# Top-level CMakeLists.txt:
cmake_minimum_required(VERSION 3.20)
project(MyProject LANGUAGES CXX)

add_subdirectory(lib)
add_subdirectory(src)
add_subdirectory(tests)

Each add_subdirectory includes its own CMakeLists.txt. Targets defined in any subdirectory are visible to all others (within the same CMake project). This makes it easy to structure large projects into logical modules.

Compilation Databases (compile_commands.json)

A compilation database is a JSON file that records the exact compiler command used for each source file. It is consumed by tools like clang-tidy, clangd (the language server), cppcheck, and IDEs to understand how your code is compiled without re-running CMake.

# Generate compile_commands.json with CMake:
cmake -S . -B build -DCMAKE_EXPORT_COMPILE_COMMANDS=ON

# The file looks like:
[
  {
    "directory": "/home/user/project/build",
    "command": "g++ -std=c++20 -O2 -I/home/user/project/include -c main.cpp -o main.o",
    "file": "/home/user/project/main.cpp"
  },
  {
    "directory": "/home/user/project/build",
    "command": "g++ -std=c++20 -O2 -I/home/user/project/include -c engine.cpp -o engine.o",
    "file": "/home/user/project/engine.cpp"
  }
]

Many developers symlink compile_commands.json from the build directory to the project root so that clangd and VS Code find it automatically:

ln -s build/compile_commands.json compile_commands.json

Without a compilation database, language servers cannot know which include paths, macros, or standards your project uses. The result is false errors, missing intellisense, and incorrect go-to-definition. Always generate and maintain a compilation database.

Key insight: compile_commands.json is the bridge between your build system and your development tools (linters, language servers, static analyzers). It ensures your editor understands the same compilation context as your build.

Ninja: The Fast Build Executor

While Make is the traditional build executor, Ninja was designed from the ground up for speed. Created by Evan Martin (a Chrome developer), Ninja is intentionally minimal: it does not have a scripting language, conditionals, or pattern rules. It only executes build graphs as fast as possible.

# Ninja is typically generated by CMake, not written by hand:
cmake -S . -B build -G Ninja
cmake --build build   # uses ninja internally

# Ninja advantages over Make:
# - Faster dependency checking (uses a compact binary log)
# - Better parallelism (builds all ready targets simultaneously)
# - Simpler, less overhead per command
# - No tab-sensitivity issues

For large projects (Chromium, LLVM, Android), Ninja reduces build times significantly compared to Make. Most modern C++ projects use CMake + Ninja as their build stack.

Package Managers: vcpkg and Conan

C++ has historically lacked a standard package manager. Developers downloaded libraries manually, built them from source, and configured include/library paths by hand. Modern package managers automate this process.

vcpkg

vcpkg (by Microsoft) is a C++ package manager that integrates seamlessly with CMake. It downloads, builds, and installs open-source libraries from a central registry:

# Install vcpkg:
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg && ./bootstrap-vcpkg.sh

# Install packages:
./vcpkg install boost-filesystem openssl fmt spdlog

# Use with CMake (toolchain file):
cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=/path/to/vcpkg/scripts/buildsystems/vcpkg.cmake

# In CMakeLists.txt, find_package works as usual:
find_package(fmt CONFIG REQUIRED)
find_package(spdlog CONFIG REQUIRED)
target_link_libraries(app PRIVATE fmt::fmt spdlog::spdlog)

vcpkg supports a manifest mode where dependencies are declared in a vcpkg.json file that lives in your project repository:

{
    "name": "my-project",
    "version": "1.0.0",
    "dependencies": [
        "fmt",
        "spdlog",
        { "name": "boost-filesystem", "version>=": "1.80.0" },
        "openssl"
    ]
}

Conan

Conan is a decentralized C++ package manager that supports multiple build systems and binary caching:

# Install Conan:
pip install conan

# Create a conanfile.txt:
[requires]
fmt/10.1.0
spdlog/1.12.0
boost/1.83.0

[generators]
CMakeDeps
CMakeToolchain

# Install dependencies:
conan install . --output-folder=build --build=missing

# Configure and build:
cmake -S . -B build --preset conan-release
cmake --build build

Conan's key advantage is binary caching: once a package is compiled for a specific configuration (compiler, architecture, build type), the binary is cached and reused. This dramatically speeds up CI/CD pipelines where dependencies are installed on every build.

vcpkg vs Conan

Both are production-ready. vcpkg is the default choice for CMake-centric projects. Conan is preferred when you need binary caching or support for non-CMake build systems.

How Incremental Builds Work

The core logic of incremental building is dependency graphs and file timestamps. When you modify a file, the build system determines which outputs are affected and rebuilds only those:

  1. You edit engine.h.
  2. The build system checks the dependency graph: engine.cpp, main.cpp, and physics.cpp all include engine.h.
  3. All three object files (engine.o, main.o, physics.o) are outdated.
  4. The build system recompiles those three files (in parallel, if possible).
  5. The executable depends on the object files, so the linker runs again.
  6. Other files that do not include engine.h are untouched.

This is why header design matters so much for build performance. If engine.h is included by 200 source files, changing it triggers 200 recompilations. If you use the PIMPL idiom (discussed in the Translation Units article) to hide implementation details, a change to the implementation header triggers recompilation of only one file.

▶ Dependency Graph: What Recompiles After a Header Change?

Step through a header modification to see how the build system determines which files need recompilation.

Build Types and Configurations

CMake supports multiple build configurations, each with different compiler flags:

# Single-config generators (Makefiles, Ninja):
cmake -S . -B build-debug -DCMAKE_BUILD_TYPE=Debug
cmake -S . -B build-release -DCMAKE_BUILD_TYPE=Release

# Multi-config generators (Visual Studio, Xcode, Ninja Multi-Config):
cmake -S . -B build -G "Ninja Multi-Config"
cmake --build build --config Debug
cmake --build build --config Release

The standard configurations and their typical flags:

You can also define custom build type flags:

# Enable sanitizers in Debug mode:
if(CMAKE_BUILD_TYPE STREQUAL "Debug")
    target_compile_options(app PRIVATE -fsanitize=address,undefined)
    target_link_options(app PRIVATE -fsanitize=address,undefined)
endif()

CMake Presets

CMake 3.19+ supports presets: named configurations stored in CMakePresets.json. Presets standardize how the project is configured and built across different developers and CI systems:

{
    "version": 6,
    "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 },
    "configurePresets": [
        {
            "name": "dev-debug",
            "displayName": "Development Debug",
            "generator": "Ninja",
            "binaryDir": "${sourceDir}/build/debug",
            "cacheVariables": {
                "CMAKE_BUILD_TYPE": "Debug",
                "CMAKE_EXPORT_COMPILE_COMMANDS": "ON",
                "CMAKE_CXX_STANDARD": "20"
            }
        },
        {
            "name": "release",
            "displayName": "Release",
            "generator": "Ninja",
            "binaryDir": "${sourceDir}/build/release",
            "cacheVariables": {
                "CMAKE_BUILD_TYPE": "Release",
                "CMAKE_CXX_STANDARD": "20",
                "CMAKE_INTERPROCEDURAL_OPTIMIZATION": "ON"
            }
        }
    ],
    "buildPresets": [
        { "name": "dev-debug", "configurePreset": "dev-debug" },
        { "name": "release", "configurePreset": "release" }
    ]
}
# Use presets:
cmake --preset dev-debug     # configure
cmake --build --preset dev-debug   # build

Other Build Systems

While CMake dominates, several other build systems are worth knowing:

Meson

Meson is a modern build system that uses Python-like syntax and generates Ninja files. It is faster to configure than CMake and has cleaner syntax, but a smaller ecosystem:

# meson.build
project('myapp', 'cpp', default_options: ['cpp_std=c++20'])
engine_lib = static_library('engine', 'engine.cpp', 'physics.cpp')
executable('app', 'main.cpp', link_with: engine_lib)

Bazel

Bazel (by Google) is a monorepo-oriented build system focused on hermetic builds and massive-scale caching. It excels at large organizations with thousands of developers but has a steeper learning curve:

# BUILD
cc_binary(
    name = "app",
    srcs = ["main.cpp"],
    deps = [":engine"],
)
cc_library(
    name = "engine",
    srcs = ["engine.cpp", "physics.cpp"],
    hdrs = ["engine.h", "physics.h"],
)

xmake

xmake is a Lua-based build system with built-in package management, popular in the Chinese C++ community:

-- xmake.lua
add_rules("mode.debug", "mode.release")
set_languages("c++20")
target("app")
    set_kind("binary")
    add_files("*.cpp")

C++20 Modules: Rewriting the Build Model

C++20 introduces modules, which fundamentally change how C++ code is organized and compiled. Modules replace the #include mechanism with a compiled, binary interface:

// engine.cppm - module interface unit
export module engine;
import <string>;
import <vector>;

export class Engine {
    std::string name_;
    std::vector<int> data_;
public:
    Engine(std::string name);
    void run();
};

// engine.cpp - module implementation unit
module engine;
Engine::Engine(std::string name) : name_(std::move(name)) {}
void Engine::run() { /* ... */ }

// main.cpp - consumer
import engine;
int main() {
    Engine e("demo");
    e.run();
}

How Modules Change the Build

The Build Challenge

Modules introduce a new kind of dependency that did not exist before: the build order of module interface units matters. If main.cpp imports module engine, then engine.cppm must be compiled first to produce the BMI. The build system must understand module dependencies to schedule compilations correctly.

This is fundamentally different from the #include model, where all TUs can be compiled in parallel because headers are just text files. With modules, the build system must scan import declarations to build a dependency graph of module units before it can start compiling.

CMake 3.28+ supports C++20 modules natively:

cmake_minimum_required(VERSION 3.28)
project(MyApp LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

add_library(engine)
target_sources(engine
    PUBLIC FILE_SET CXX_MODULES FILES engine.cppm
    PRIVATE engine.cpp
)

add_executable(app main.cpp)
target_link_libraries(app PRIVATE engine)

Current Status

As of early 2026, module support varies by compiler:

Build system support is the bigger bottleneck. CMake, Ninja, and Visual Studio handle modules, but the ecosystem (package managers, CI tools, code analyzers) is still catching up. For new projects, modules are worth adopting. For existing large codebases, migration is a significant effort best done incrementally.

Key insight: Modules are not just a faster #include. They change the fundamental build model from "compile everything in parallel, link at the end" to "compile in dependency order, with module interfaces as checkpoints." Build systems must understand this ordering to schedule builds correctly.

Build Acceleration: ccache and sccache

ccache caches compilation results. If you recompile a file with the same source content and flags, ccache returns the cached result instantly instead of running the compiler.

# Install and use ccache:
sudo apt install ccache

# Configure CMake to use ccache:
cmake -S . -B build -DCMAKE_CXX_COMPILER_LAUNCHER=ccache

# Or set globally:
export CXX="ccache g++"

ccache is transformative for development workflows: switching branches, clean builds after pulling, and CI pipelines all benefit. sccache (by Mozilla) adds distributed caching: compilation results are shared across a team via a cloud storage backend (S3, GCS, Azure Blob Storage).

Build System Best Practices

  1. Use CMake + Ninja as your default build stack. It works on every platform and IDE.
  2. Use modern target-based CMake. Set properties on targets, not global variables.
  3. Generate compile_commands.json always. Your editor and linting tools depend on it.
  4. Use a package manager (vcpkg or Conan) instead of manually managing third-party libraries.
  5. Enable ccache in development. Enable sccache in CI.
  6. Use precompiled headers for stable, frequently included headers.
  7. Keep headers lean and use forward declarations to minimize the dependency graph.
  8. Enable LTO for release builds (CMAKE_INTERPROCEDURAL_OPTIMIZATION).
  9. Use CMake presets to standardize build configurations across your team.
  10. Evaluate C++20 modules for new projects or performance-critical subsystems.

Summary