git fetch: Cannot allocate memory

Question

First I do: git submodule update --init

    Cloning into '_aaa'...
    Checking connectivity... done.
    Submodule path '_aaa': checked out '87311dd0bb0ac9b77cd9bbac0a15671858f0cf17'

then: git fetch --recurse-submodules

    Fetching submodule _base
    Fetching submodule _aaa
    Auto packing the repository for optimum performance. You may also
    run "git gc" manually. See "git help gc" for more information.
    Counting objects: 21678, done.
    Compressing objects: 100% (20602/20602), done.
    Writing objects: 100% (21678/21678), done.
    Total 21678 (delta 8973), reused 0 (delta 0)
    Removing duplicate objects: 100% (256/256), done.
    fatal: Out of memory? mmap failed: Cannot allocate memory
    fatal: Out of memory? mmap failed: Cannot allocate memory
    error: failed to run prune

Last update was pretty big about 1GB but there was no files more then 40MB in.

I increased amount of memory on the server from 512M up to 1024M but the problem still exists anyway. I also tried different plays with git gc, window, compress and so on without any result.

as a alternative solution you could fetch the submodules one at a time instead of using `--recurse-submodules` — gitaarik, Dec 18 '13 at 08:22
Did you take a look at this question yet? http://stackoverflow.com/q/6775242/1248175 — gitaarik, Dec 18 '13 at 08:29
Of course but they as I understand has a different situation like: git fetch --work and big files I have no one and git version 1.8.4.1. — user1934268, Dec 18 '13 at 08:34

mockinterface · Accepted Answer · 2013-12-18T11:27:45.917

3

The error itself originates in git's sha1 object handling code, but it is hard to say what's going on exactly without say knowing what's the errno returned by mmap.

Could you run your command under strace and post here the lines around mmap failure?

EDIT: Try git config --add core.bigFileThreshold 4m (or maybe even a lesser number).

core.bigFileThreshold

    Files larger than this size are stored deflated, without attempting delta compression. Storing large files without delta compression avoids excessive memory usage, at the slight expense of increased disk usage.

    Default is 512 MiB on all platforms. This should be reasonable for most projects as source code and other text files can still be delta compressed, but larger binary media files won’t be.

    Common unit suffixes of k, m, or g are supported.

https://www.kernel.org/pub/software/scm/git/docs/git-config.html

edited Dec 18 '13 at 11:27

answered Dec 18 '13 at 08:56

mockinterface

14,452
5
28
49

Here it is the first income

...
[pid 23816] fcntl(3, F_SETFD, FD_CLOEXEC) = 0
[pid 23816] read(3, "PACK\0\0\0\2\0\0T\256", 12) = 12
[pid 23816] lseek(3, 918541449, SEEK_SET) = 918541449
[pid 23816] read(3, "n\373\t\n\341\330Tf\253\372\263/\211e%\f8\5Y(", 20) = 20
[pid 23816] mmap(NULL, 918541469, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)
[pid 23816] mmap(NULL, 918541469, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)
[pid 23816] write(2, "fatal: Out of memory? mmap faile"..., 58) = 58
[pid 23816] exit_group(128)             = ?

– user1934268 Dec 18 '13 at 09:42

Interesting. I have no clear answer, it just looks that plainly it runs out of memory. Could it be that you just have a huge "/idol/.client/.git/modules/_aaa/packed-refs" file? It's just a text file to speed up git lookup - instead of looking for the objects in the file system. Also, you said that you played with git gc - did you try `git gc --aggressive --prune=now`? – mockinterface Dec 18 '13 at 10:37
git gc --aggressive --prune=now Counting objects: 118, done. Compressing objects: 100% (115/115), done. Writing objects: 100% (118/118), done. Total 118 (delta 48), reused 70 (delta 0) – user1934268 Dec 18 '13 at 10:52
git fetch --recurse-submodules Fetching submodule _base Fetching submodule _aaa fatal: Out of memory? mmap failed: Cannot allocate memory – user1934268 Dec 18 '13 at 10:52
/idol/.client/.git/modules/_aaa/packed-refs # pack-refs with: peeled fully-peeled 2fa2b7ba4de47bf0867732cf345e662bc654c96f refs/heads/master 2fa2b7ba4de47bf0867732cf345e662bc654c96f refs/remotes/origin/master – user1934268 Dec 18 '13 at 10:53
You mean that these are the only lines in the packed-refs file? Let's also try setting `core.bigfilethreshold` to avoid mapping the files into memory - will require a fairly recent git version. – mockinterface Dec 18 '13 at 11:12
Magic, it works. Thanks a lot. Command after git config: .git/modules/_aaa _aaa; git submodule update --init; git fetch --recurse-submodules – user1934268 Dec 18 '13 at 11:47

git fetch: Cannot allocate memory

1 Answers1

Linked