,

Contents · ABI, calling conventions, linking


What is an ABI?

  • Application Binary Interface defines binary contracts between components.
  • Includes data layout, endianness, calling conventions, syscalls, and object formats.
  • Stable ABIs enable interoperability across toolchains and languages.

Calling conventions

  • x86-64 System V vs Windows x64: argument registers, callee/caller-saved sets.
  • Return values in registers; varargs handling and red zones (SysV).
  • Special cases: vector/FP args, struct returns, tail calls.
SysV: rdi,rsi,rdx,rcx,r8,r9  |  Win64: rcx,rdx,r8,r9; 128-byte shadow space

Stack frames and prologue/epilogue

  • Prologue sets up stack frame, saves callee-saved regs; epilogue restores and returns.
  • Alignment requirements (e.g., 16-byte for SSE on x86-64).
  • Exception unwinding metadata and frame pointers vs frame chaining.
push rbp; mov rbp,rsp; sub rsp,N; ... ; leave; ret

Static vs dynamic linking

  • Static: resolves at build time, larger binaries, simpler deployment.
  • Dynamic: shared libraries resolved at load/run time; smaller binaries, updatable.
  • Loader duties: relocation, symbol resolution, TLS setup.

ELF/PE/Mach-O, symbols, and relocations

  • Sections and segments; symbol tables; relocation entries and types.
  • ELF: .plt, .got, .rela; PE: import table, delay loads; Mach-O: stubs and bind info.
  • Position-independent code and relocation models (small/large).

PLT/GOT, TLS, and position-independent code

  • PLT (procedure linkage table) lazy binds external calls via GOT entries.
  • Thread-Local Storage models: local-exec, initial-exec, general-dynamic.
  • PIE/PIB: text relocation avoidance and ASLR compatibility.
call foo@PLT → resolver → GOT[foo] patched to direct address

Versioning, symbol visibility, and compatibility

  • Symbol versioning (ELF) and DLL exports (PE); hidden vs default visibility.
  • ABI breaks: data layout changes, vtable reordering, calling convention shifts.
  • Stability strategies: C ABIs for FFI, opaque handles, semver discipline.

Exercises

  1. Disassemble a function compiled for SysV and Win64; compare prologue/epilogue and arg passing.
  2. Build a small shared library and inspect its symbols and relocations with objdump/llvm-readobj.
  3. Write a minimal PIC function and trace its PLT/GOT resolution at runtime.
ABIs are the contracts that let binaries interoperate—understand them to build stable libraries and performant runtimes.