How to perform shallow clone using Composer?

Question

I've got the following composer.json:

{
    "require": {
        "php": ">=5.2.0",
        "queueit/KnownUser.V3.PHP": "dev-master"
    },
    "config": {
        "autoloader-suffix": "ComposerManager",
        "vendor-dir": "../../../all/libraries/composer"
    },
    "prefer-stable": true,
    "repositories": [
        {
            "type": "package",
            "package": {
                "name": "queueit/KnownUser.V3.PHP",
                "version": "dev-master",
                "source": {
                    "type": "git",
                    "url": "https://github.com/queueit/KnownUser.V3.PHP.git",
                    "reference": "master"
                }
            }
        }
    ]
}

However when I run:

$ composer -vvv update
...
Cloning master
Executing command (CWD): git clone --no-checkout 'https://github.com/queueit/KnownUser.V3.PHP.git' '.../sites/all/libraries/composer/queueit/KnownUser.V3.PHP' && cd '.../sites/all/libraries/composer/queueit/KnownUser.V3.PHP' && git remote add composer 'https://github.com/queueit/KnownUser.V3.PHP.git' && git fetch composer

the cloning process takes very long time and the repository grows over 25MB in size:

$ du -hs ~/.composer/cache/vcs/https---github.com-queueit-KnownUser.V3.PHP.git/
25M ~/.composer/cache/vcs/https---github.com-queueit-KnownUser.V3.PHP.git/

Then the Composer stops with the timeout:

[Symfony\Component\Process\Exception\ProcessTimedOutException]
The process "git clone --no-checkout 'https://github.com/queueit/KnownUser.V3.PHP.git' '.../sites/all/libraries/composer/queueit/KnownUser.V3.PHP' && cd '.../sites/all/libraries/composer/queueit/KnownUser.V3.PHP' && git remote add composer 'https://github.com/queueit/KnownUser.V3.PHP.git' && git fetch composer" exceeded the timeout of 300 seconds.

I assume the repository is too big to clone all the git objects.

How I can clone the repository quicker by using a shallow clone?

For example by passing extra --depth 1 or --single-branch git parameters into Git command, so it can be automatically picked up by Composer?

I expect the change to be self-contained within the composer.json file, so no external configuration should be required when invoked this file on the other systems or by other users when running composer install.

score 5 · Accepted Answer · answered Nov 17 '17 at 12:57

Shallow cloning using Composer isn't officially supported (without any patching) as the git parameters are hardcoded.

There is already a feature request to add this: Add support for git shallow clones. However implementing this feature can cause some issues (such as not reaching the locked commits if the depth is not so high^@stof and other).

Furthermore there is a pull request which attempts to implement shallow clones by adding an extra --git-clone-depth parameter (tests shows some good results). However the change has been abandon due to faster git clones using cache.

For the quick hack, it's possible to edit git parameters in doDownload() in src/Composer/Downloader/GitDownloader.php, e.g. by changing --depth 1 --single-branch in this line:

$command = 'git clone --no-checkout ...'

Or find the way to apply depth 1 setting into git config.

The easiest workaround (without any hacks) for bigger repositories is just to increase the timeout by specifying the variable like:

COMPOSER_PROCESS_TIMEOUT=0 composer install

How to perform shallow clone using Composer?

1 Answers1