Subproject dependencies in pants

Question

I am new to using pantsbuild and I can't seem to find any good questions, answers, or documentation around my dillema.

I have a Pants project which should be buildable on its own. It has its own pants and pants.ini file as well as all BUILD files containing paths relative to the project root (where pants.ini is). This project is hosted on GitHub.

I'd like to use this project as a dependency in a second project. I've chosen to use git submodules to do this. Now, I have a layout like the following:

path
├── pants
├── pants.ini
├── projectA
│   └── src
│       └── python
|           └── main
│               ├── BUILD
│               └── main.py
└── projectB
    ├── pants
    ├── pants.ini
    └── src
        └── python
            ├── libA
            |   ├── BUILD
            |   └── lib.py
            └── libB
                ├── BUILD
                └── lib.py

Naturally, I am looking to use projectB's BUILD targets from within projectA, so in projectA's BUILD, I have something of the sort:

dependencies = [ "projectB/src/python:libA" ]

This is all well and good. However, since projectB is an independent project, it's src/python/libA/BUILD file contains something of the sort:

dependencies = [ "src/python:libB" ]

Because of this, projectB can indeed be built independently. However, when trying to build projectA, the build targets from projectB search starting from projectA's project root, for instance:

Exception Message: libB was not found in BUILD files from path/src/python

Does pantsbuild have any clean way to handle these subproject dependencies? Or would I be forced to alter the BUILD files of the subproject in order to fit them within my project layout (Causing the project to be unbuildable independently)?

Any solutions or advice is welcomed!

score 1 · Accepted Answer · answered Apr 18 '17 at 22:34

So it turns out that the functionality I was hoping for was not supported by pants. At first, I followed the advice of a Yi Cheng in the comments and created a script to replace all subproject BUILD rules with those that are in relation to the root project. That script can be found here: https://github.com/brandonio21/pants-subproject-prep

The script was pretty limited, however, and was far from a workable solution. I have since changed pants upstream to support the behavior. The PR was here: https://github.com/pantsbuild/pants/pull/4088

Now, if you have a subproject within another pants project, building the root project with --subproject-roots=["path/to/subproject1_root", "path/to/subproject2_root"] (Or specifying the option in pants.ini) will build the subproject from the proper subproject root

score 0 · Answer 2 · answered Oct 16 '16 at 05:21

0

This is all well and good. However, since projectA is an independent project, it's src/python/libA/BUILD file contains something of the sort:

dependencies = [ "src/python:libB" ]

iiuc src/python:libB needs to be projectB/src/python:libB. All target paths in the repo should be relative to the build root which is path in your example.

answered Oct 16 '16 at 05:21

yi cheng

193
10

This is correct. In order for the project to work properly, projectB/src/python/libA/BUILD should contain `dependencies = [ "projectB/src/python:libB" ]` – brandonio21 Oct 16 '16 at 05:40
1

Woops, pressed enter too early. However, my point is, projectB is an independent project. I would like it to be buildable on its own without any knowledge of the fact that it is in another project. – brandonio21 Oct 16 '16 at 05:42
Just noticed there are two sets of `pants` and `pants.ini`, but only one should be allowed at the build root. e.g. in projectB, you would have to convert target paths w.r.t to the new build root. The behavior is also funky here because pants determines build root based on where `pants` executable is located. – yi cheng Oct 16 '16 at 05:47
Right. I was hoping there might be some way to overcome this, especially since projectB does not have any dependencies on projectA. – brandonio21 Oct 16 '16 at 05:55
I do not see any shortcuts per se, but it should be relatively easy to write a script replacing `src/python` with `projectB/src/python` in BUILD files. – yi cheng Oct 16 '16 at 06:12

Subproject dependencies in pants

2 Answers2