Mmap(2) is atomic with respect to the mappings across all threads; in part, at least, because unmap(2) also is. To break it down, the scenario described looks something like:
MapRegion(from, to, obj) {
Lock(&CurProc->map)
while MapIntersect(&CurProc->map, from, to, &range) {
MapUnMap(&CurProc->map, range.from, range.to)
MapObjectRemove(&CurProc->map, range.from, range.to)
}
MapInsert(&CurProcc->map, from, to, obj)
UnLock(&CurProc->map)
}
Following this, map_unmap has to ensure that while it is removing the mappings, no thread can access them. Notice the Lock(&thisproc->map).
MapUnMap(map, from, to) {
foreach page in map.mmu[from .. to] {
update page structure to invalidate mapping
}
foreach cpu in map.HasUsed {
cause cpu to invoke tlb cache invalidation for (map, from, to)
}
}
The first phase is to re-write the processor specific page tables to invalidate the area(s).
The second phase is to force every cpu that has ever loaded this map into its translation cache to invalidate that cache. This bit is highly architecture dependent. On an older x86, rewriting cr3 is typically enough, so the HasUsed is really CurrentlyUsing; whereas a newer amd64 might be able to cache multiple address space identifiers, so would be HasUsed. On an ARM, local tlb invalidation is broadcast to the local cluster; so HasUsed would refer to cluster ids rather than cpu ones. For more detail, search for tlb shootdown, as this is colloquially known as.
Once these two phases are complete, no thread can access this address range. Any attempt to do so will cause a fault, which will cause the faulting thread to Lock its mapping structure, which is already locked by the mapping thread, so it will wait until the mapping is complete. When the mapping is complete, all of the old mappings have been removed and replaced by new mappings, so there is no way to retrieve a previous mapping after this point.
What if another thread references the address range during the update? It will either continue with stale data or fault. In this respect stale data isn't an inconsistency, it is as if it had been referenced just before the mapping thread had entered mmap(2). The faulting case is the same as for faulting thread above.
In summary, update to the mappings is implemented using a series of transactions which ensure a consistent view of the address space. The cost of these transactions is architecture specific. The code to implement this can be quite intricate as it needs to guard against implicit operations, such as speculative fetching, as well as explicit ones.
