• Hierarchy overview and AMAT
• L1/L2/L3 properties and inclusivity
• TLBs, page size, huge pages
• Hardware/software prefetching
• Bandwidth vs latency, roofline
• NUMA basics and policies
• Placement, first-touch, affinity
• Measuring and profiling
• Exercises

• CPU accesses progress from registers → L1 → L2 → LLC (L3) → DRAM → storage.
• Average Memory Access Time (AMAT) models latency contributions across levels.
• Workloads vary: compute-bound vs memory-bound; reuse and locality dominate.

AMAT ≈ L1_hit + L1_miss_rate × (L2_hit + L2_miss_rate × (L3_hit + L3_miss_rate × DRAM))

• L1: per-core, small and fast; L2: larger, per-core; L3: largest, often shared.
• Inclusive LLC simplifies snooping; exclusive reduces duplication; non-inclusive is flexible.
• Prefetchers exist at multiple levels and can affect inclusivity behavior.

• Translation Lookaside Buffers cache page table entries; TLB misses are costly.
• Use larger pages (2MB, 1GB) for streaming/large arrays to reduce TLB pressure.
• Beware of fragmentation and reduced protection granularity.

TLB reach = (#entries) × (page size)

• Hardware: next-line, stride, stream, spatial; tune via MSRs/BIOS where available.
• Software: compiler hints and intrinsics (e.g., __builtin_prefetch, _mm_prefetch).
• Balance pollution and timeliness; prefetch distance depends on latency and ILP.

// Software prefetch hint (GCC/Clang)
for (size_t i = 0; i < n; ++i) {
  __builtin_prefetch(&a[i + 64], 0, 3);
  sum += a[i];
}

• Compute throughput vs memory bandwidth determines upper bounds.
• Arithmetic intensity (FLOPs/byte) positions kernels on the roofline.
• Vectorization and tiling raise intensity; streaming stores save bandwidth.

• Non-Uniform Memory Access: each socket has local memory; remote access is slower.
• Policies: default (first-touch), interleave, preferred, bind; affect placement and latency.
• Tools: numactl, lscpu, lstopo (hwloc) to inspect and control locality.

# Run on NUMA node 0 with interleaved memory
numactl --cpunodebind=0 --membind=0 ./app

• Initialize memory on the threads that will use it (first-touch policy).
• Pin threads to cores/sockets to preserve locality and reduce migration.
• Use partitioned data structures to reduce cross-socket sharing.

# Pin to physical cores 0-7
sudo taskset -c 0-7 ./app

• Measure misses (L1/L2/L3), bandwidth, latency, TLB stats with perf/VTune/pcm.
• Trace prefetchers; tune BIOS/MSR controls cautiously, validate with benchmarks.
• NUMA counters: local vs remote accesses, inter-socket traffic.

1. Implement cache tiling for a stencil and compare AMAT-derived predictions to measurements.
2. Enable huge pages and measure TLB miss rate changes for a streaming workload.
3. Optimize an app for NUMA: add thread pinning and first-touch initialization; measure speedup.