Curriculum
Intro to SME
Two chapters. Chapter 1 is five steps through a GEMM: hardware, the inner pipeline, the microkernel, the roofline, and memory in flight. Chapter 2 is pack, softmax, and one head of attention. A linear-algebra self-check sits in front if matrix multiply is rusty.
M=4 · N=4 · K=2 · SVL=128
Two chapters
The self-check is optional. Chapter 1 stays five steps. Chapter 2 starts after memory in flight.
Linear algebra self-check
Chapter 1 · GEMM
Five steps, one shape: M=4, N=4, K=2, SVL=128.
-
01
Hardware · Live
The hardware
A register, the ZA tile, and the compute unit — three names every other page uses.
Open this pageZ · ZA · ×+
-
02
SME · Live
Kernel tour
whilelt, ld1w, fmopa, st1w — the GEMM inner loop, in the order it runs.
Start this tourwhilelt → ld1w → fmopa → st1w
-
03
SME · Live
Microkernel
How pack, the K loop, and the store compose a real GEMM kernel — in every transpose state.
Play this kernelpack → ldr + fmopa ×2 → st1w / stnt1w
-
04
Performance model · Live
Roofline
Why reuse, not just FLOPs, decides speed — the memory wall in one picture.
Open this pageperformance = min(peak, supply × intensity)
-
05
Cache choreography · Live
Memory in flight
Prefetch and store traffic across two kernels, level by level — and the one slot where HBM reads and writes collide.
Open this pagepldl2keep / pldl1strm / stnt1w — across kernels
Chapter 2 · Beyond GEMM
Pack bridges out of the microkernel. Softmax is the step that is not a GEMM. Attention composes both.
-
01
Beyond GEMM · Live
Pack
An 8×8 row-major panel becomes four contiguous 4×4 runs.
Play this kernel8×8 row-major → four 4×4 runs
-
02
Beyond GEMM · Live
Softmax
Subtract the max before the exp, so a hot score stays finite and the weights still sum to 1.
Play this kernelexp(x − max) / Σ exp
-
03
Beyond GEMM · Live
Attention
One query, four keys: scores, softmax weights, then a mix with V.
Play this kernelQKᵀ → softmax → ×V