Gradle vs Bazel performance of builds

Question

So everyone is talking about bazel now, but migration to it isn't automated (gradle is better in this regard when migrating from maven). So I don't want to invest time into manually converting any repository to it.

But I can't find any information about build speed difference between recent versions of gradle (5.6 >) and bazel (1.0).

Can anyone share a link or his own experience? I'm mostly interested in incremental builds where just a couple of files have changed.

Jin · Answer 1 · 2019-11-03T21:30:37.873

9

Dropbox ran some benchmarks recently:

Note that the clean build without a cache scenario is significantly slower for Bazel, whereas the incremental build scenario is significantly faster.

Bazel build units tend to have a smaller granularity than Gradle's. It's common to see java_library targets with just a single source file. The command line actions in these build units, also known as targets, are individually cached (locally or remotely) and composed together to build a java_binary.

With many small build units, there are typically more actions to execute, and therefore more disk I/O and computation, leading to a slower initial, clean build time.

Some executables of these actions may also have a high startup cost (e.g javac), which adds up when these processes are restarted many times. Bazel has a mechanism called persistent workers, where the executable process for individual actions (e.g. a compiler wrapper for javac, tsc, scalac or ghc) can be persisted across action executions, saving on startup times and enabling an even lower level of caching at the process level.

On the other hand, Bazel's small build units enable highly incremental builds and fast iterative development, as seen in the chart above.

The small build units also enable Bazel to form a dependency graph with a high degree of parallelism. With remote execution, you can run 100s and 1000s of small build actions in parallel.

The dependency graph is also highly optimized for the no-op build case. If nothing has changed in your project, Bazel should take as little time as possible to figure out nothing has changed, so nothing needs to be done.

The drawback of a slow clean build can also be mitigated with remote caches, remote execution, or not running the relatively rare bazel clean since builds aim to be hermetic, deterministic, and consistent. A build with 100% remote cache hits are common with Bazel.

edited Nov 03 '19 at 21:30

answered Nov 02 '19 at 02:34

Jin

12,748
3
36
41

1

There's no point in this comparison because the most important part is gradle with build cache vs bazel with build cache and incremental compilation. As to granularity of the build - I don't see how this matters. Gradle supports incremental compilation of java down to files level. You would need to find which files have changed across the whole project anyway, right? Could you elaborate on that? It also isn't clear which version of gradle they using. So I have serious doubts about that this comparisons done properly... – user1685095 Nov 02 '19 at 22:08
2

And no op buid of 28.6 seconds? They probably aren't using gradle properly if they have this configuration times. – user1685095 Nov 02 '19 at 22:09
I dont think that they are doing proper use of gradle either. Build times are crazy – cocorossello Nov 03 '19 at 15:00
1

The more granular a build unit is, the better a build system is able to parallelize the work across execution threads, and minimize the work that needs to be done. You can do a comparison between a java_library with 100 Java source files, and 10 java_library targets with 10 source files each that's connected in a DAG. With the latter, changing a Java source file often doesn't trigger a recompilation of 100 source files every single time. The difference is even more apparent if you modify non-ABI parts of a Java file. – Jin Nov 03 '19 at 21:22
1

As for Gradle, I don't have insight into how Dropbox structured their build. You can try out a benchmark comparison yourself with https://github.com/jin/android-projects - my personal benchmark results corroborate. – Jin Nov 03 '19 at 21:23
I've tried to play with your android-projects and made changes in `java_only` modules to test incremental build. In every case gradle was faster, both in incremental build test and cached build (I've done a git restore). I guess your benchmark is just outdated because you were using gradle 4.12 and there were a lot of improvements in gradle 5.0 and higher. But just judging from gradle team benchmarks - they say that 1000 modules java project could be built within 3 seconds in incremental build. So I'm pretty sure there's no peformance benefit from using bazel. – user1685095 Nov 04 '19 at 14:02

user1685095 · Accepted Answer · 2020-05-11T19:22:10.573

Okay, I've migrated closed source project containing ~100_000 loc of java to bazel.

Gradle 5.6.3 vs bazel 1.0.1

As I've said my interest is about developer productivity and hence about incremental builds.

I've used the following in BUILD for bazel so I haven't used most granular approach because I don't think it's really maintainable and that anybody except big companies with a lot of devops could use that. And in any case I see the fact that I would need to do this manually to improve build speed as a downside of bazel.

java_binary(
    srcs = glob(["src/main/java/**/*.java"]),
    resources = glob(["src/main/resources/**"]),
    ...
)

Incremental build with one or two changed files.

gradle - 1s, bazel - 4.227s

I've tried that multiple times and each time gradle was significatly faster. I Haven't tested incremental build when more than one or two files have been changed, maybe in that scenario bazel would be the same or faster than gradle.

No op build gradle - 700ms, bazel - 0.090ms

So speed-wise gradle seems to be the winner for developer productivity. Bazel have more safe defaults (error-prone is on by default) in gradle you have to enable it yourself, but IMHO the flexibility of gradle outweighs more safe defaults of bazel.

By not making the Bazel build granular you are basically negating the biggest benefit of Bazel. If you had more granularity that incremental build time should go WAY down. — aljoscha, Nov 27 '19 at 13:35
@aljoscha I know and I think that any sane modern build system should do that for me automatically. I should only say that I want to compile java project. See shake, meson even redo. — user1685095, Nov 27 '19 at 13:55

Gradle vs Bazel performance of builds

2 Answers2