The meaning of the modulo operation is the standard mathematical meaning, x mod y
is the rest of x div y
, where div
is the integer division.
Modulus naturally arises when there is a periodicity over the natural numbers, to see this, just take compute x mod 5
for the first 30 numbers.
Alignment is a requirement that an address is multiple of some number n, since x is multiple of n iif x mod n == 0, this links the modulus with the alignment.
The modulus over the naturals is one operation where it is easy to visualize the quotient group: when aligning an address at the n-boundary, there are n possible cases to considers, despite a possibly infinite set of addresses.
For example, to align and address at the 4-boundary, addresses like 1, 5, 9, 13, ... are all equivalent since they are one value above a multiple of 4.
For all of them, we just need to add 3.
So, for an alignment at the n-boundary we only consider the cases where the address is 0, 1, 2, ... n-1 bytes above a multiple of n.
The wording "ESP = 12 modulo 16" means that esp
is 12 bytes above an address multiple of 16.
Note that being 12 bytes above a multiple of 16 is equal to being 4 bytes below the next multiple of 16.
Visualizing 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
we can go to the right by adding 1 and to the left by subtracting 1.
The reason esp
is 12 modulo 16 after a call is that if, when an instruction like call function
is executed, the stack is aligned on 16 bytes (i.e. esp
is 0 modulo 16) then immediately after the call the stack is 4 bytes lower1 (due to the call
instruction pushing the return address) and thus 12 bytes from the previous multiple of 16.
... 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 ...
^ ^-- Before the call
|__ After the call
1 Remember, the stack grows down.