How to write files to current directory instead of bazel-out

Question

I have the following directory structure:

my_dir
|
 --> src
|    |
|     --> foo.cc
|     --> BUILD
|
 --> WORKSPACE
|
 --> bazel-out/ (symlink)
| 
| ...

src/BUILD contains the following code:

cc_binary(
    name = "foo",
    srcs = ["foo.cc"]
)

The file foo.cc creates a file named bar.txt using the regular way with <fstream> utilities.

However, when I invoke Bazel with bazel run //src:foo the file bar.txt is created and placed in bazel-out/darwin-fastbuild/bin/src/foo.runfiles/foo/bar.txt instead of my_dir/src/bar.txt, where the original source is.

I tried adding an outs field to the foo rule, but Bazel complained that outs is not a recognized attribute for cc_binary.

I also thought of creating a filegroup rule, but there is no deps field where I can declare foo as a dependency for those files.

How can I make sure that the files generated by running the cc_binary rule are placed in my_dir/src/bar.txt instead of bazel-out/...?

@JakeFreeman It's just a regular directory in user's home. Why does that matter? — Ani Kristo, Dec 12 '17 at 01:58
@JakeFreeman Bazel creates a symlink `bazel-out/` that points to `/private/var/tmp/_bazel_//execroot//bazel-out/` — Ani Kristo, Dec 12 '17 at 02:01
@JakeFreeman I could do that, but it feels a bit hacky. I suppose Bazel should have some sort of support for this, as it is a fairly common use-case. But it's missing in the documentation. — Ani Kristo, Dec 12 '17 at 02:23
Using relative path is an integral part of C++ and should not be considered hacking if anything it is better. — Jake Freeman, Dec 12 '17 at 02:51
This is the way Bazel is designed: it does not modify/write to your source/input directories from which your build rule inputs are defined in. — Jin, Dec 12 '17 at 04:33
Thanks for the answer @Jin I am curious to know what is the reason for this design choice? Or, what could be a good way or a good practice of how to handle the output files that the code generates? — Ani Kristo, Dec 16 '17 at 12:00

Jin · Accepted Answer · 2017-12-19T15:52:02.450

Bazel doesn't allow you to modify the state of your workspace, by design.

The short answer is that you don't want the results of the past builds to modify the state of your workspace, hence potentially modifying the results of the future builds. It'll violate reproducibility if running Bazel multiple times on the same workspace results in different outputs.

Given your example: imagine calling bazel run //src:foo which inserts

#define true false
#define false true

at the top of the src/foo.cc. What happens if you call bazel run //src:foo again?

The long answer: https://docs.bazel.build/versions/master/rule-challenges.html#assumption-aim-for-correctness-throughput-ease-of-use-latency

Here's more information on the output directory: https://docs.bazel.build/versions/master/output_directories.html#documentation-of-the-current-bazel-output-directory-layout

score 4 · Answer 2 · edited Dec 30 '21 at 15:56

4

There could be a workaround to use genrule. Below is an example that I use genrule to copy a file to the .git folder.

genrule(
    name = "precommit",
    srcs = glob(["git/**"]),
    outs = ["precommit.txt"],
    # folder contain this BUILD.bazel file is tool which will be symbol linked, we use cd -P to get to the physical path
    cmd = "echo 'setup pre-commit.sh' > $(OUTS) && cd -P tools && ./path/to/your-script.sh",
    local = 1,  # required
)

edited Dec 30 '21 at 15:56

joel

6,359
2
30
55

answered Sep 13 '19 at 05:29

Syi

108
1
6

Thanks a lot! cd -P is a great tool to fight with Bazel :) – Victor Jun 10 '21 at 17:57
Lol that's cheating – Jasha Sep 14 '22 at 03:22

score 0 · Answer 3 · answered Nov 21 '19 at 19:21

If you're passing the name of the output file in when running, you can simply use absolute paths. To make this easier, you can use the realpath utility if you're in linux. If you're on a mac, it is included in brew install coreutils. Then running it looks something like:

bazel run my_app_dir:binary_target -- --output_file=`realpath relative/path/to.output

score 0 · Answer 4 · answered Nov 10 '21 at 10:10

This has been discussed and explained in a Bazel issue. Recommendation is to use a tool external to Bazel:

As I understand the use-case, this is out-of-scope for building and in the scope of, perhaps, workspace configuration. What I'm sure of is that an external tool would be both easier and safer to write for this purpose, than to introduce such a deep design change to Bazel.

The tool would copy the files from the output tree into the source tree, and update a manifest file (also in the source tree) that lists the path-digest pairs. The sources and the manifest file would all be versioned. A genrule or a sh_test would depend on the file-generating genrules, as well as on this manifest file, and compare the file-generating genrules' outputs' digests (in the output tree) to those in the manifest file, and would fail if there's a mismatch. In that case the user would need to run the external tool, thus update the source tree and the manifest, then rerun the build, which is the same workflow as you described, except you'd run this tool instead of bazel regenerate-autogenerated-sources.

How to write files to current directory instead of bazel-out

4 Answers4

Linked