Same native code rebuilt for every build variant

Question

My build.gradle file builds some native code:

externalNativeBuild {
    ndkBuild {
        path 'src/main/cpp/Android.mk'
    }
}

The problem is that I have tens of build variants, and every time I switch to a different variant the same native code is built again (into a different folder), and this slows down a lot my build time.

What's the best way to build the native code only once and share it between different build flavours?