Why isn't Rust linker faster with incremental builds?

Question

I'm new to Rust and I'm trying to understand why can't Rust build faster. I'm specifically talking about the most common case, where I've made a small change in one of my source files and then I need to wait seconds for the cargo build to do its job. Even if my app code is very small, if I add dependencies on MySQL and Rocket for example, those two crates will come with their own dependencies and apparently that's what makes the build process considerably slower. Obiously it is quite subjective what "slow" means, but if I need to wait 5-10 seconds for something I do a hundred times a day, I want to know why that needs to be the case and is there something I'm missing.

It seems to me that Cargo doesn't waste much time on the check whether it needs to re-compile any of those 200-300 (sub-)dependencies, but it is the linker that takes so long. And I'm trying to understand if there is an objective reason why the linker can't somehow optimize that process. Isn't it possible to somehow build and cache the whole MySQL and Rocket dependencies into two bigger libs, for example, and avoid doing all that work every single time? Even at the cost of some code duplication in the binary.

BTW I tried the LLD linker (I'm using Ubuntu) and it speeds things up to some degree, but it still seems to be heavily impacted by the big number of sub-dependencies.

I suggest having a look at: https://fasterthanli.me/articles/why-is-my-rust-build-so-slow, but yes. Rust compile time is a bit slower due to monomorphizing and the fact that the crates you pull in can greatly affect what kind of compile-time you get. — Anton Blomström, Jan 14 '22 at 17:39

score 1 · Answer 1 · answered Apr 28 '23 at 06:42

Isn't it possible to somehow build and cache the whole MySQL and Rocket dependencies into two bigger libs, for example, and avoid doing all that work every single time?

We can link statically compiled libraries of MySQL/Rocket into our Rust project (By a C++ wrapper). Let's try it.

Create build.rs file, we'll use the cxx crate to compile the MySQL and Rocket dependencies into static libraries.

extern crate cxx_build;

fn main() {
    cxx_build::bridge("src/wrapper.rs")
        .file("src/mysql_wrapper.cpp")
        .file("src/rocket_wrapper.cpp")
        .flag("-std=c++17")
        .compile("mysql_wrapper", "rocket_wrapper");
}

Create wrapper files called wrapper.rs, mysql_wrapper.cpp and rocket_wrapper.cpp in our src directory.

// mysql_wrapper.cpp

#include <mysql.h>

void init_mysql() {
    mysql_library_init(0, NULL, NULL);
}

void cleanup_mysql() {
    mysql_library_end();
}

// rocket_wrapper.cpp

#include <rocket.h>

#include <iostream>

extern "C" {
    void init_rocket() {
        std::vector<std::string> args = { "myapp", "--port=8000" };
        rocket::config::Config config;
        config.from_args(args);
        rocket::ignite(config);
    }

    void stop_rocket() {
        rocket::shutdown();
    }
}

// wrapper.rs

#[cxx::bridge]
mod ffi {
    unsafe extern "C++" {
        fn init_mysql();
        fn cleanup_mysql();
        fn init_rocket();
        fn stop_rocket();
    }
}

Modify the Rust code (main.rs) to link against the static libraries instead of the MySQL and Rocket dependencies directly.

// main.rs

mod ffi {
    #[cxx::bridge]
    mod ffi {
        unsafe extern "C++" {
            fn init_mysql();
            fn cleanup_mysql();
            fn init_rocket();
            fn stop_rocket();
        }
    }
}

fn main() {
    unsafe {
        ffi::init_mysql();
    }
    // Usage of MySQL lib here
    unsafe {
        ffi::cleanup_mysql();
    }

    unsafe {
        ffi::init_rocket();
    }
    // Usage of Rocket f/work here
    unsafe {
        ffi::stop_rocket();
    }
}

Build the project

cargo build

This is it. This will include mysql and rocket as a static library and will not be linking from Cargo.toml as a direct dependency. I'm not sure if we'll get a really significant gain, it'll probably depend on the project structure, and should only be used if you don't plan to update the versions of these dependencies. I'll probably have to do a performance check, but later.

Why isn't Rust linker faster with incremental builds?

1 Answers1

Linked