Let's number the 32 digits of an int, from 0 to 31. Digit #0 represents 2⁰, digit #1 represents 2¹, etc. More generally, digit #n represents 2ⁿ.
But we only have digits up to #31: we have 32 digits, but we started counting at 0. There is no 32nd digit to create 2³².
So what does the computer do? It basically pretends you wrote 2³² but then drops all but the first 32 digits (digits 0-31). This is called integer overflow. That means that 2³², which "should" be 1 followed by 32 0s, is actually just 32 0s -- which, of course, equals 0. Subtract 1 from that, and you get -1.
If you're working with longs (not ints), then you have 64 bits to work with instead of 32. In that case, overflow doesn't happen at bit 32, and you'll find that 2³² - 1 doesn't equal -1: it equals 4294967295, just like you'd expect. But in that case, 2⁶⁴ is 0 for similar reasons.