Course Homepage

Build Automation

Building software

How do you build a C program?

gcc -o hello hello.c

What if I have two C files?

gcc -o main main.c hello.c

What if I have 30,000+ C files?

gcc -o vmlinux kernel/locking/mutex-debug.c \
  kernel/locking/rwsem.c kernel/locking/rtmutex.c \
  kernel/locking/qrwlock.c kernel/locking/irqflag-debug.c \
  kernel/locking/test-ww_mutex.c kernel/locking/mutex.c \
  # 30,000+ more

Why not build the whole program at once?

git clone https://git.kernel.html/pub/scm/linux/kernel/git/torvalds/linux.git
find kernel/ arch/ block/ crypto/ drivers/ fs/ init/ ipc/ lib/ mm net/ security/ sound/ virt/ -name "*.c" | wc -l

Why not build all at once

Separate compilation

This is how C/C++ do separate compilation, but other languages, like Java, further incorporate language abstractions with source code organization, such as Java’s use of one-class per file.

main.c     -> gcc -> main.o
square.c   -> gcc -> square.o
                        v
                       ld
                        v
                       main

Example

main.c

#include <stdio.h>
#include "square.h"

int main(void) {
    int x = 2;
    printf("The square of %d is %d\n", x, square(x));
}

square.c

int square(int x) { return x * x; }

square.h

int square(int x);

How to do separate compilation?

gcc -c main.c

-c compiles the .c file to an object file .o

Object files are translations of C to machine code.

Without -c, gcc will also link the C runtime and standard libraries, which is why trying to compile without main will throw a linker error.

We will cover how source code becomes a program and and running process later in the semester.

Compiling our example

Compile each .c file to a .o file (without linking)

gcc -c main.c  # -c flag produces .o files
gcc -c square.c

Then link all object files into one program binary

gcc -o main main.o square.o # pass .o files

Notice gcc takes the .o files instead of

gcc looks at file extensions to distinguish source code from object files.

gcc will run linking for you, but you can also run ld the linker yourself. We will cover this more later in the semester.

Incremental builds

Only rebuild source files that changed.

vim main.c    # Modify one source file
gcc -c main.c # Recompile it

Re-link with existing object files

gcc -o main main.o square.o

https://nvd.nist.gov/vuln/detail/CVE-2024-3094

https://www.openwall.com/lists/oss-security/2024/03/29/4

https://infosec.exchange/@fr0gger/112189232773640259

https://arstechnica.com/security/2024/03/backdoor-found-in-widely-used-linux-utility-breaks-encrypted-ssh-connections/

https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/

Build automation

Makefiles

target … : prerequisites …
        recipe
        …
        …

https://www.gnu.html/software/make/manual/make.html#Rule-Introduction

Basic makefile

main:
    gcc -o main main.c square.c

Won’t rebuild if C file is changed.

Basic with clean

main:
    gcc -o main main.c square.c

clean:
    rm -f main

Dependencies

main: main.c square.c
    gcc -o main main.c square.c

Will rebuild if C file is changed, but everything is recompiled.

Incremental build

main: main.o square.o
    gcc -o main main.o square.o

main.o: main.c
    gcc -c main.c

square.o: square.c
    gcc -c square.c

Only rebuilds the C file that changed!

make
touch main.c
make

Wildcard patterns

main: main.o square.o
    gcc -o main main.o square.o

%.o: %.c
    gcc -c $<

Variables

PROG := main
SRC = main.c square.c
OBJ = $(SRC:%.c=%.o)

$(PROG): $(OBJ)
    $(CC) $(CFLAGS) -o $@ $^

%.o: %.c
    $(CC) $(CFLAGS) -c -o $@ $<

Phony targets

PROG := main
SRC = main.c square.c
OBJ = $(SRC:%.c=%.o)

.PHONY: all clean

all: $(PROG)

$(PROG): $(OBJ)
    $(CC) $(CFLAGS) -o $@ $^

%.o: %.c
    $(CC) $(CFLAGS) -c $<

clean:
    $(RM) $(PROG) $(OBJ)

Targets are files.

What if want a special target that doesn’t create a file?

“all” is a convention. First target is always the default.

“clean” target is another convention for removing generated files.

When we get to version control, convention is to only push non-generated files. Ship build automation script instead of binaries.

More than incremental builds

SolarWinds

https://cloud.google.com/blog/topics/threat-intelligence/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor

https://cloud.google.com/blog/topics/threat-intelligence/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor

https://cloud.google.com/blog/topics/threat-intelligence/unc2452-merged-into-apt29

https://arstechnica.com/information-technology/2020/12/russian-hackers-hit-us-government-using-widespread-supply-chain-attack/

https://arstechnica.com/information-technology/2020/12/only-an-elite-few-solarwinds-hack-victims-received-follow-on-attacks/

https://arstechnica.com/information-technology/2020/12/security-firm-fireeye-says-nation-state-hackers-stole-potent-attack-tools/

hello project

hello project