Why does Mutex contain a Box?

Question

Rust's std::sync::Mutex is a struct that contains a heap-allocated inner mutex, along with this semi-mysterious comment:

pub struct Mutex<T: ?Sized> {
    // Note that this mutex is in a *box*, not inlined into the struct itself.
    // Once a native mutex has been used once, its address can never change (it
    // can't be moved). This mutex type can be safely moved at any time, so to
    // ensure that the native mutex is used correctly we box the inner mutex to
    // give it a constant address.
    inner: Box<sys::Mutex>,
    poison: poison::Flag,
    data: UnsafeCell<T>,
}

The comment explains that Box is used to give the inner mutex a stable address. But I can't seem to find any explanation for why a stable address is required in the first place.

On Unix-like platforms at least, the "native mutex" here (sys::Mutex) is ultimately a wrapper around libc::pthread_mutex_t (source code).

In C, it almost makes sense to have a rule against moving mutexes, because mutexes are used through pointers, and moving one while there is a live pointer to it would be clearly wrong. But in Rust, you can't even try to move something unless there are no live references to it. So this line of argument doesn't seem convincing.

Why must the native mutex have a stable address?

I suppose it's to allow move of rust without problem if LLVM copy the value. — Stargateur, Jul 18 '18 at 01:37
Also [Can a pthread_mutex_t be moved in memory?](https://stackoverflow.com/q/14614523/155423). Empahsis mine: *Some mutex implementations on Linux, for example, use the futex system call which specifically waits on **the address of the mutex*** — Shepmaster, Jul 18 '18 at 01:49
@Shepmaster that seems like a good reason why `Mutex` mustn't be movable _while it's locked_. But it won't be movable while locked in any case, because `MutexGuard` borrows it for the duration. Does the futex limitation affect unlocked mutexes also? — Jack O'Connor, Jun 10 '21 at 19:34
@Jack I feel that the second answer to the duplicate is the most direct answer to that. That is, when you create a mutex, you must initialize it *before* you lock it, and when you move it, it becomes de-initialized, so it would have to be re-initialized before it can be locked again. Does looking at it that way help? I can relate to the feeling that that answer isn't really satisfactory (because it doesn't explain why initialization would be needed anyway, if the address of the mutex is what matters). — trent, Jun 10 '21 at 21:36
@trentcl I asked the same question to Mara Bos on Twitter (https://twitter.com/m_ou_se/status/1403413967846977538), and she said something similar. In short, pthread_mutex_t makes no guarantees about its implementation, which means it _might_ hold self-referential pointers (in the same way that GCC's std::string holds a self-referential pointer when it's short). I suppose in the end this doesn't actually have much to do with locking at all. It's just that _any_ opaque C type might do this sort of thing and might not be safe to move as a result. — Jack O'Connor, Jun 11 '21 at 21:21
@Shepmaster I think this question should be reopened, since it's not as closely related to the question about std::mutex as it might've seemed at first. Specifically: In C++, std::mutex needs to worry about being moved _while it's locked_. That means that it needs to worry about the behavior of the underlying system calls, like futex on Linux. (And indeed, since futex takes the address of the lock as an argument, we definitely can't move the lock around while it's locked.) But this specific issue doesn't apply in Rust. Instead, Rust's concern is that `pthread_mutex_t` _may_ be self-referential — Jack O'Connor, Jun 11 '21 at 21:27
(To add to all the above, I suppose self-referential pointers aren't the only weird thing that `pthread_mutex_t` might choose to do. There might also be e.g. some global registry of all locks, and it could be the case that `pthread_mutex_init` stashes a pointer in that registry. Again, in theory any opaque C type might do this sort of thing.) — Jack O'Connor, Jun 11 '21 at 21:33
@JackO'Connor sure, go ahead and vote to reopen. I didn’t vote to close this, so I don’t particularly have a strong opinion. — Shepmaster, Jun 11 '21 at 21:41
@JackO'Connor note that the *original question asker* voted to close it (that’s what “community” means in the close list) so I’d tend to defer to them in most cases (also because trentcl knows what they are talking about). — Shepmaster, Jun 11 '21 at 21:45
Hmm, I think I agree with what you're saying, @Jack. Will you write that answer? — trent, Jun 11 '21 at 22:02

Jack O'Connor · Accepted Answer · 2023-07-11T01:14:13.467

I think the answer here is essentially just "pthread_mutex_t is an opaque type". The POSIX standard doesn't guarantee that it's movable, so we can't make any assumptions about it. Mara Bos from the Rust library team discussed this in a Twitter thread.

There are many ways that a C type might not be movable. It might contain a self-referential pointer, i.e. one of its fields might point into another of its fields. This isn't usually allowed in Rust, but it is allowed in C and C++, and some real world types do this as an optimization. (For example, GCC's implementation of std::string does this.) Another thing a C type might do, is to insert a pointer to itself into some global registry. In either of these cases, moving or copying an instance of such a type would lead to dangling pointers and other inconsistencies.

Some (most?) implementations of pthread_mutex_t don't do these things. But the POSIX standard doesn't prohibit them, and any given implementation could start doing these things in the future without warning. The Rust standard library has to be defensive about this, and that's why Mutex always needs a Box on Unix. (Note that Mutex has changed a bit since this question was written. It no longer contains a Box on Windows or Linux, but it does on other Unix variants.)

As an aside, it's also the case that the futex system call (which is how pthread_mutex_t is implemented on Linux) takes the address of the lock as an argument. This sort of thing is another reason that pthread_mutex_t isn't movable in general. However, this particular issue doesn't concern Rust, because a Mutex in Rust is always borrowed while it's locked, and that means it can only move while it's unlocked.

Why does Mutex contain a Box?

1 Answers1