So here is what I found after looking at some of the the source code (MVC4).
http://aspnetwebstack.codeplex.com/SourceControl/changeset/view/553690ac9488#src%2fWebMatrix.WebData%2fExtendedMembershipProvider.cs
SimpleMembership is an implementation of the abstract class ExtendedMembershipProvider.
The code inside SimpleMembership is mostly SQL operations and some calls to the underlying (called "previous" in the documentation) MembershipProvider.
I don't think it is of any use (in my case) to override SimpleMembership as its implementation is mostly tied to SQL Server. Instead, for what I understand, I should implement ExtendedMembershipProvider. Then, by setting this implementation in the web.config file, the WebSecurity helper would bypass SimpleMembership (default implementation) and call my implementation of the ExtendedMembershipProvider.
I don't think I will do this any soon since it looks even more complicated than before (more methods to implement)... but still doable.
However, all this said, I'm a bit disappointed that we still have to work with the MembershipProvider which, IMHO, is far (a lot of static and internal stuff) from the whole dependency injection thing that we love so much with ASP.Net MVC/WebApi.
Edit 1
This question was aked before Jon Galloway wrote this tutorial :
http://weblogs.asp.net/jgalloway/archive/2012/08/29/simplemembership-membership-providers-universal-providers-and-the-new-asp-net-4-5-web-forms-and-asp-net-mvc-4-templates.aspx
But my answer stays valid as this (taken from Jon Galloway article) resumes it:
Note that SimpleMembership still requires some flavor of SQL Server -
it won't work with MySQL, NoSQL databases, etc. You can take a look at
the code in WebMatrix.WebData.dll using a tool like ILSpy if you'd
like to see why - there are places where SQL Server specific SQL
statements are being executed, especially when creating and
initializing tables. It seems like you might be able to work with
another database if you created the tables separately, but I haven't
tried it and it's not supported at this point.