Build Automation
Tools
COP-3402
Table of Contents
Building software
How do you build a C program?
gcc -o hello hello.c
What if I have two C files?
gcc -o main main.c hello.c
What if I have 30,000+ C files?
gcc -o vmlinux kernel/locking/mutex-debug.c \ kernel/locking/rwsem.c kernel/locking/rtmutex.c \ kernel/locking/qrwlock.c kernel/locking/irqflag-debug.c \ kernel/locking/test-ww_mutex.c kernel/locking/mutex.c \ # 30,000+ more
Why not build the whole program at once?
git clone https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git find kernel/ arch/ block/ crypto/ drivers/ fs/ init/ ipc/ lib/ mm net/ security/ sound/ virt/ -name "*.c" | wc -l
Downsides of building all at once
- Long compilation time
- Small changes require complete recompile
- Intentionally organize your code into separate parts
- Learn this in software engineering
- Projects won't be that large in this class
- I'll give you the organization for the compiler project
Separate compilation
- Divide program into separate C files
- Compile individually
- Link compiled files
This is how C/C++ do separate compilation, but other languages, like Java, further incorporate language abstractions with source code organization, such as Java's use of one-class per file.
Diagram
main.c -> gcc -> main.o square.c -> gcc -> square.o exponent.c -> gcc -> exponent.o v ld v main
Example
main.c
#include <stdio.h> int square(int); int main() { printf("%d\n", square(3)); }
square.c
int exponent(int x, int y); int square(int x) { return exponent(x, 2); }
exponent.c
int exponent(int x, int y) { int result = 1; for (; y > 0; result *= x, y--); return result; }
How do we separately compile this program?
gcc -c main.c
-c
compiles the .c
file to an object file .o
Object files are translations of C to machine code.
Without -c
, gcc will also link the C runtime and standard libraries, which is why trying to compile without main will throw a linker error.
We will cover how source code becomes a program and and running process later in the semester.
Compiling our example
Compile each .c file (without linking) to a .o object file
gcc -c main.c # -c flag produces .o files gcc -c square.c gcc -c exponent.c
Then link all object files into a single program binary
gcc -o main main.o square.o exponent.o # pass .o files
Notice gcc takes the .o files instead of
gcc looks at file extensions to distinguish source code from object files.
gcc will run linking for you, but you can also run ld
the linker yourself. We will cover this more later in the semester.
Incremental builds
Only rebuild source files that changed.
emacs main.c # modify one source file gcc -c main.c # recompile it
Re-link with existing object files
gcc -o main main.o square.o exponent.o
Build automation
Makefiles
target … : prerequisites … recipe … …
Basic makefile
main: gcc -o main main.c square.c exponent.c
Won't rebuild if C file is changed.
Basic with clean
main: gcc -o main main.c square.c exponent.c clean: rm -f main
Dependencies
main: main.c square.c exponent.c gcc -o main main.c square.c exponent.c
Will rebuild if C file is changed, but everything is recompiled.
Incremental build
main: main.o square.o exponent.o gcc -o main main.o square.o exponent.o main.o: main.c gcc -c main.c square.o: square.c gcc -c square.c exponent.o: exponent.c gcc -c exponent.c
Only rebuilds the C file that changed!
make touch main.c make
Diagram
Wildcard patterns
main: main.o square.o exponent.o gcc -o main main.o square.o exponent.o %.o: %.c gcc -c $<
Variables
PROG := main SRC = main.c square.c exponent.c OBJ = $(SRC:%.c=%.o) $(PROG): $(OBJ) $(CC) $(CFLAGS) -o $@ $^ %.o: %.c $(CC) $(CFLAGS) -c -o $@ $<
Phony targets
PROG := main SRC = main.c square.c exponent.c OBJ = $(SRC:%.c=%.o) .PHONY: all clean all: $(PROG) $(PROG): $(OBJ) $(CC) $(CFLAGS) -o $@ $^ %.o: %.c $(CC) $(CFLAGS) -c $< clean: $(RM) $(PROG) $(OBJ)
Targets are files.
What if want a special target that doesn't create a file?
"all" is a convention. First target is always the default.
"clean" target is another convention for removing generated files.
When we get to version control, convention is to only push non-generated files. Ship build automation script instead of binaries.
More than incremental builds
- Run tests
- Generate documentation
- Anything you can script in bash