Maven building only changed files

Question

Lets say i have module structure like below

     Modules
       ->utils
       ->domain
       ->client
       ->services
       ->deploy (this is at the module level)

Now to lauch the client i need to make a build of all the modules, i.e utils, domain, client, services, because i am loading the jars of all the above modules to fianlly lanch the client

And all the jars gets assembled in the module deploy.

My question is if i change anything in services for example, then is there a way when running a build from deploy maven could recognise it has to build only services and hence build it and deploy it in deploy folder?

Hi, i have a doubt. If your module let's say service is using a final constant value (final int FOO=5) and it is referenced in domain module. Now Your changing the constant value as (final int FOO=10) and if you try to deploy only the service module, wouldn't this cause a issue in domain module as the constant value will be still set to previous value (FOO=5) during earlier compilation. In order to reflect the new final constant value, we need to compile the domain module also. How could we handle this use case? — Jugi, Dec 09 '19 at 17:19

score 41 · Answer 1 · edited Dec 02 '14 at 14:25

41

If you only call "mvn install" without "clean", the compiler plugin will compile only modified classes.

edited Dec 02 '14 at 14:25

user987339

10,519
8
40
45

answered Mar 14 '13 at 09:34

Puce

37,247
13
80
152

1

just use mvn compile I think it's ok, since you just want to run the project not build war and also install will take more time. – OQJF Mar 14 '13 at 09:38
1

i think you misunderstood my question...it want to build with only changed files ...i dont want to rebuild and copy the jars of unchanged modules ..? – user1224036 Mar 14 '13 at 09:39
If this doesn't work, use `mvn install -X` to discover why. In my case, our test coverage system, `clover` (which we may have misconfigured), adds modified sources to the target directory; Maven always detects them as stale, requiring recompilation. – Eponymous Apr 18 '22 at 17:23

smilyface · Answer 2 · 2021-12-26T07:10:59.637

For GIT

mvn install -amd -pl $(git status | grep -E "modified:|deleted:|added:" | awk '{print $2}' | cut -f1 -d"/")

OR

In your .bashrc file (.bashrc can be found in home directory ~/.bashrc , or create it if doesn't exists) add the following function.

mvn_changed_modules(){
    [ -z "$1" ] && echo "Expected command : mvn_changed_modules (install/build/clean or any maven command)" && exit 0

        modules=$(git status | grep -E "modified:|deleted:|added:" | awk '{print $2}' | cut -f1 -d"/")

                if [  -z "$modules" ];
                then
                        echo "No changes (modified / deleted / added)  found"
                else
                        echo -e "Changed modules are : `echo $modules`\n\n"
                        mvn $1 -amd -pl $modules
                fi
}

**Then after re-starting your bash** (command prompt), you **can just use the following command** from the ROOT directory itself.

smilyface@machine>ProjectRootDir]$ mvn_changed_module install

How it works
As per the question mvn install -amd -pl services is the command when "some changes done in services module". So, first get module name from the changed file(s) and put it as input for mvn-install command

Say for example, below is a list of modified files (output of git status) -
services/pom.xml
services/ReadMe.txt
web/src/java/com/some/Name.java
Then services and web are the modules name which need to be build / compile / install

This did not work for me because my mvn (3.8.1) expects the -pl parameters to be seperated by comma, I guess. Here is what worked instead on bash@macos --> mvn install -amd -pl $(git status | grep -E "modified:|deleted:|added:" | awk '{print $2}' | cut -f1 -d"/" | sort |uniq| paste -sd "," -) — ᐅdevrimbaris, Jul 30 '21 at 07:50
@ᐅdevrimbaris, Can I update my answer with this (for mac)? — smilyface, Dec 26 '21 at 07:09

score 15 · Answer 3 · answered Mar 14 '13 at 12:55

15

Within a multi-module build you can use:

mvn -pl ChangedModule compile

from the root module will compile only the given ChangedModule. The compiler plugin will only compile the files which have been changed. But it can happen that the module you have changed would cause a recompile of other module which are depending on the ChangedModule. This can be achieved by using the following:

mvn -amd -pl ChangedModule compile

where the -amd means also make dependents. This will work without installing the whole modules into the local repository by a mvn install.

answered Mar 14 '13 at 12:55

khmarbaise

92,914
28
189
235

Still not what i wanted, i need a jar to be built, and copied to a reporsitory (deploy folder) of only modules thats changed... – user1224036 Mar 14 '13 at 13:12
That can't accomplished with Maven. You need to write a script yourself. Or write an appropriate Maven plugin for that. That beyond the idea of maven. – khmarbaise Mar 14 '13 at 14:10

buggy · Answer 4 · 2017-01-24T11:59:31.497

After trying and using aforementioned advises, I've met following problems:

Maven install (without clean) still takes a lot of time, which for several projects can be 10-20s extra time.
Sebasjm's solution is fast and useful (I was using it for a couple of months), but if you have several changed projects, rebuilding them all the time (if you even hadn't change anything) is a huge waste of time

What really worked for me is comparing source modification dates against .jar modification in local repository. And if you check only for VCS changed files (see sebasjm's answer), then date comparison won't take noticeable time (for me it was less than 1s for 100 changed files). Main benefit of such approach is very accurate rebuild of only really changed projects. Main problem is doing modification date comparison is a bit more than one-liner script.

For those, who want to try it, but too lazy to write such script themself sharing my version of it: https://github.com/bugy/rebuilder (linux/windows). It can do some additional useful things, but the main idea and central algorithm is as explained above.

A git version would be great. – ᐅdevrimbaris Dec 27 '21 at 08:14 — ᐅdevrimbaris, Dec 27 '21 at 08:14

score 3 · Accepted Answer · answered Oct 24 '14 at 17:40

3

If you are using SVN and *nix, from the root module

mvn install -amd -pl $(svn st | colrm 1 8 | sed 's /.*  ' | xargs echo | sed 's- -,:-g' | sed 's ^ : ')

answered Oct 24 '14 at 17:40

sebasjm

108
2
6

7

Could someone add a short explanation of the thinking behind this command chain? – J Fabian Meier Oct 23 '17 at 12:08
1

everything inside `$()` is called first: **1** `svn st` to list all the modified projects **2** `colrm` remove first 8 cols **3** `sed` strip all after the first slash **4** `xargs echo` take the list and make a comma separated (instead of line-break) **5** `sed` add `:` at the end the use the list of comma separated projects with maven – sebasjm Oct 26 '17 at 19:57
@napster - As per Question (in this thread) "mvn install -amd -pl services" is the command when "some changes done in services module alone". So get changed file's module name and put it as input for mvn-install command. – smilyface Apr 23 '18 at 09:59
Try this --> mvn install -amd -pl $(git status | grep "modified:" | awk '{print $2}' | cut -f1 -d"/") – smilyface Apr 23 '18 at 10:03
The above is for modified alone.. need to change it to accept added / modified / kind of all changes. I will add it as an answer here. – smilyface Apr 23 '18 at 10:03
downvoting the answer because it has no explanation – seinecle Dec 29 '20 at 09:13

score 2 · Answer 6 · answered Sep 20 '16 at 12:00

I had the same frustration and I also wrote a project at the time - alas it is not available but I found people who implemented something similar:

for example - https://github.com/erickzanardo/maven-watcher

It uses nodejs and assumes an maven project but should work on windows and unix alike.

The idea of my implementation is to watch for changes and then compile what changed. - kind of like nodemon.

So for example

When a java file changes - I compile the module
When a class file or jar changes - I do something else (for example copy the jar under tomcat and restart tomcat)

And the two are unrelated.. so if the java compilation failed, there should be no reason for the jar file to update.. and it's quite stable.

I have used it on a project with 23K .java files and it worked smoothly.

It took the watch process a couple of seconds to start - but then it would only run if change was detected so the overall experience was nice.

The next step I intended to add is similar to your SVN support - list the modified files and use them as initialization.

Important to note - if compilation fails, it will retry on the next modification. so if you are modifying multiple jars, and the compilation fails as long as you are writing code, it will retry to compile everything on each code change until it compiled successfully.

If you'd like I can try find my old project, fix it up a bit and publish it..

score 2 · Answer 7 · answered Feb 07 '22 at 08:53

mvn clean install to run full build
mvn install to compile only changed and prepare war/jars other binaries
mvn compile to compile only changed files...

So mvn compile is the fastest. but if run/debug your project with war/jars it might not show those changes.

score 0 · Answer 8 · answered Jul 30 '20 at 22:26

The question and the answers posted so far do not take the dependency tree into account. What if the utils module is changed? We need to rebuild (retest at least) it and all the modules depending on it.

Ways to do so:

https://github.com/avodonosov/hashver-maven-plugin/
https://github.com/vackosar/gitflow-incremental-builder/
Gradle Enterprise is a commercial service which provides build cache, in particular for maven
Migrate to newer build tools like Gradle or Bazel which support build caches out of box.

Maven building only changed files

8 Answers8

Linked