5

Trying to git pull, I receive the error message:

remote: error: inflate: data stream error (incorrect data check)
remote: error: corrupt loose object '188bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7'
remote: fatal: loose object 188bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7 (stored in 
./objects/18/8bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7) is corrupt
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header

I found that this could be caused by out of memory on the server, but the server has more than 2GB free memory.

I also found that this could be solved by doing a garbage collection on the remote repo, but this also throws errors:

git gc
Counting objects: 61619, done.
Delta compression using up to 2 threads.
error: inflate: data stream error (incorrect data check)
error: corrupt loose object '188bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7'
fatal: failed to read object 188bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7: Invalid a
rgument
error: failed to run repack

So I let a file system check run on the git repo, and executing git fsck in my remote repository, I receive the following error messages:

error: inflate: data stream error (incorrect data check)
error: sha1 mismatch 188bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7
error: 188bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7: object corrupt or missing
error: inflate: data stream error (incorrect data check)
error: sha1 mismatch 9dabcbe39f25e3a79ee6f2bd1598999ea2c45de5
error: 9dabcbe39f25e3a79ee6f2bd1598999ea2c45de5: object corrupt or missing
Checking object directories: 100% (256/256), done.
error: index CRC mismatch for object b2e2916e7daeb3a4a9187774d9a19a29b5f3dce1 fr
om ./objects/pack/pack-d77b3f792b3870c0c121223a58776ef537871b41.pack at offset 3
8157638
error: inflate: data stream error (incorrect data check)
error: cannot unpack b2e2916e7daeb3a4a9187774d9a19a29b5f3dce1 from ./objects/pac
k/pack-d77b3f792b3870c0c121223a58776ef537871b41.pack at offset 38157638
Checking objects: 100% (156533/156533), done.
Checking connectivity: 61617, done.
dangling commit d6251eef8e78c1592174c10004fc20e031b34be1
missing blob 188bf75e9cfba7e0a93fa3a15bdcce6c1b8858c7
missing blob 9dabcbe39f25e3a79ee6f2bd1598999ea2c45de5

What does git try to tell me and how can I fix this? How, if at all, can I prevent such repo corruptions in the future?

Alexander
  • 19,906
  • 19
  • 75
  • 162
  • 1
    This analogy fails if you push it too hard, but: think of it as Git having detected that your disk (or other storage media, if you use something other than a disk drive) is on fire. *Git* can do nothing at all about this. Git needs reliable storage: it can check *whether* the storage *is* reliable, and you can use a duplicate repository if the storage has failed, but you need some kind of reliable storage somewhere. – torek Aug 16 '17 at 15:02
  • @torek Are you saying that the LVM storage our git repo is hosted on is failing? That would be bad news indeed. – Alexander Aug 16 '17 at 15:08
  • @torek I executed fsck on the logical disk, but it found no error. – Alexander Aug 16 '17 at 15:17
  • Well, there's the analogy falling apart already. :-) It's not clear *what* damaged the repository: it may not be the underlying hardware, but rather some other software messing with file contents. All Git knows is that the bits it wrote earlier don't match the bits it's reading back now based on the built-in hash checking. – torek Aug 16 '17 at 16:36

1 Answers1

0

We finally got to the root cause of the issue a few weeks ago. After all disks were changed and the error still happened, we finally moved everything off the affected physical machine. Memtest86 then told us where to find the culprit. Not sure how this can happen on ECC memory without being detected, but then, I am not an admin.

Alexander
  • 19,906
  • 19
  • 75
  • 162