7

I'm trying to push some changes, but git push hangs. When I run git push, I see no output, and nothing seems to be happening. There's no activity in top, and no sign of anything happening.

I do not control the git hosting service. I'm using a HTTPS URL. I believe the hosting service is using dumb HTTPS, not git's "smart-HTTP" protocol. On the client side, I use Mac OS X, and I've got git 1.8.1.1 installed via Homebrew (but using the version of git included in Xcode's command-line tools doesn't seem to make a difference). Logging out and logging back in doesn't seem to help. I can pull and push to this hosting service/repository from a different Linux box.

Below is some debugging output that shows git push hanging after the client issues a PROPFIND request, gets a HTTP/1.1 100 Continue response from the server, and then nothing happens: it's just stuck.

How do I get this working? Are there any troubleshooting steps that I can try?

$ GIT_CURL_VERBOSE=1 git push -v
Pushing to https://secure2.svnrepository.com/redacted/redacted/
* About to connect() to secure2.svnrepository.com port 443 (#0)
*   Trying 67.228.18.88...
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*    subject: C=US; OU=Domain Control Validated; CN=secure2.svnrepository.com
*    start date: 2012-01-09 16:16:59 GMT
*    expire date: 2015-02-09 02:52:45 GMT
*    subjectAltName: secure2.svnrepository.com matched
*    issuer: O=AlphaSSL; CN=AlphaSSL CA - G2
*    SSL certificate verify ok.
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

< HTTP/1.1 401 Authorization Required
< Date: Wed, 23 Jan 2013 03:29:36 GMT
< Server: Apache/2.2.3 (Red Hat)
< WWW-Authenticate: Basic realm="redacted"
< Content-Length: 493
< Content-Type: text/html; charset=iso-8859-1
< 
* Ignoring the response-body
* Connection #0 to host secure2.svnrepository.com left intact
* Issue another request to this URL: 'https://secure2.svnrepository.com/redacted/redacted/info/refs?service=git-receive-pack'
* Re-using existing connection! (#0) with host (nil)
* Connected to (nil) (67.228.18.88) port 443 (#0)
* Server auth using Basic with user 'redacted'
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
Authorization: Basic redacted=
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

< HTTP/1.1 200 OK
< Date: Wed, 23 Jan 2013 03:29:36 GMT
< Server: Apache/2.2.3 (Red Hat)
< Last-Modified: Wed, 23 Jan 2013 03:00:40 GMT
< ETag: "143802e-3b-e6374600"
< Accept-Ranges: bytes
< Content-Length: 59
< Content-Type: text/plain; charset=UTF-8
< 
* Connection #0 to host (nil) left intact
* Re-using existing connection! (#0) with host (nil)
* Connected to (nil) (67.228.18.88) port 443 (#0)
* Server auth using Basic with user 'redacted'
> GET /redacted/redacted/HEAD HTTP/1.1
Authorization: Basic redacted=
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

< HTTP/1.1 200 OK
< Date: Wed, 23 Jan 2013 03:29:36 GMT
< Server: Apache/2.2.3 (Red Hat)
< Last-Modified: Wed, 16 Jan 2013 21:05:31 GMT
< ETag: "d1802c-17-3d0d7cc0"
< Accept-Ranges: bytes
< Content-Length: 23
< Content-Type: text/plain; charset=UTF-8
< 
* Connection #0 to host (nil) left intact
* About to connect() to secure2.svnrepository.com port 443 (#0)
*   Trying 67.228.18.88...
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*    subject: C=US; OU=Domain Control Validated; CN=secure2.svnrepository.com
*    start date: 2012-01-09 16:16:59 GMT
*    expire date: 2015-02-09 02:52:45 GMT
*    subjectAltName: secure2.svnrepository.com matched
*    issuer: O=AlphaSSL; CN=AlphaSSL CA - G2
*    SSL certificate verify ok.
> PROPFIND /redacted/redacted/ HTTP/1.1
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Depth: 0
Content-Type: text/xml
Content-Length: 181
Expect: 100-continue

< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 401 Authorization Required
< Date: Wed, 23 Jan 2013 03:29:37 GMT
< Server: Apache/2.2.3 (Red Hat)
< WWW-Authenticate: Basic realm="redacted"
< Content-Length: 493
< Content-Type: text/html; charset=iso-8859-1
* the ioctl callback returned 0
< 
* Ignoring the response-body
* Connection #0 to host secure2.svnrepository.com left intact
* Issue another request to this URL: 'https://secure2.svnrepository.com/redacted/redacted/'
* Re-using existing connection! (#0) with host (nil)
* Connected to (nil) (67.228.18.88) port 443 (#0)
* Server auth using Basic with user 'redacted'
> PROPFIND /redacted/redacted/ HTTP/1.1
Authorization: Basic redacted=
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Depth: 0
Content-Type: text/xml
Content-Length: 181
Expect: 100-continue

< HTTP/1.1 100 Continue

I don't have strace on my Mac OS X machine, and I can't figure out how to use dtruss to see what system calls it is hanging on because dtruss requires me to be root, and then git push will work differently.

Update: I've reproduced this on a Linux machine with git 1.8.1.4 and with strace. Running strace shows something like the following before it hangs:

sendto(4, <redacted>..., 314, 0, NULL, 0) = 314
recvfrom(4, "\27\3\1\0000", 5, 0, NULL, NULL) = 5
recvfrom(4, "E\202\271\21\236p\200\346\374\3641\355\t\275\rLi\202T)\326\271l/\351\f\357\2769Jb\22"..., 48, 0, NULL, NULL) = 48
select(5, [4], [4], [], {0, 729000}) = 1 (out [4], left {0, 728997})
poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}, {fd=4, events=POLLOUT|POLLWRNORM}], 2, 0) = 1 ([{fd=4, revents=POLLOUT|POLLWRNORM}])
select(5, [4], [], [], {0, 729000}) = 0 (Timeout)
poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
...last 2 lines repeat infinitely...

So it seems to be hanging expecting to receive something from the server.

Also, I tried a similar trace with GIT_CURL_VERBOSE=1 git push -v on an older Linux box running git 1.7.4.4, and it starts with the same prefix and then continues on from there. On the broken machine with the newer git:

$ grep '^> [A-Z]' git-1.8.1.1-trace.stderr
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
> GET /redacted/redacted/HEAD HTTP/1.1
> PROPFIND /redacted/redacted/ HTTP/1.1
> PROPFIND /redacted/redacted/ HTTP/1.1

On the older machine with the older git, where it's all working:

$ grep '^> [A-Z]' git-1.7.4.4-trace.stderr
> GET /g_wagner/c79-s13/info/refs?service=git-receive-pack HTTP/1.1
> GET /g_wagner/c79-s13/info/refs?service=git-receive-pack HTTP/1.1
> GET /g_wagner/c79-s13/HEAD HTTP/1.1
> PROPFIND /g_wagner/c79-s13/ HTTP/1.1
> PROPFIND /g_wagner/c79-s13/ HTTP/1.1
> HEAD /g_wagner/c79-s13/info/refs HTTP/1.1
> HEAD /g_wagner/c79-s13/objects/info/packs HTTP/1.1
> MKCOL /g_wagner/c79-s13/info/ HTTP/1.1
> LOCK /g_wagner/c79-s13/info/refs HTTP/1.1
> GET /g_wagner/c79-s13/objects/info/packs HTTP/1.1
...
> UNLOCK /g_wagner/c79-s13/info/refs HTTP/1.1

Looking at the full trace on both machines, I can't see any difference in what is sent in the problematic PROPFIND request (the 2nd PROPFIND): both requests appear to be identical, except for the User-Agent: header.

D.W.
  • 3,382
  • 7
  • 44
  • 110
  • `< HTTP/1.1 401 Authorization Required` looks like an authentication issue. – cjc343 Jan 23 '13 at 20:32
  • Thanks, @cjc343. Any tips on how to troubleshoot it further? I am using `~/.netrc` for authentication, and I've confirmed that my `~/.netrc` is exactly identical to its value on another (Linux) machine where I have no problem pushing. Also I am able to pull successfully from this machine, which would also require authentication credentials -- so it's all very puzzling. – D.W. Jan 23 '13 at 21:57
  • It's certainly odd... unfortunately I'm not very familiar with git over http/s as I've always used ssh for authentication. Unless pull permissions accidentally got left open it makes no sense that you can't push, and throws most of the possibilities, like permissions for `.netrc` being too open or a username included in the remote (which I'd think would show up above if it were the case), out the window. If you add a remote that does have the username included, does git prompt you for the password when pushing (it should)? Hopefully someone else has a better idea of what's going wrong... – cjc343 Jan 23 '13 at 22:37
  • It's clear that it's trying to find git-http-backend, failing, and falling back to DAV. But are you sure that DAV is actually supported? It seems like this might just be a read-only access method. – hobbs Jun 15 '13 at 02:07
  • @hobbs, beats me! How would I tell? Again, on a client running git 1.7.4.4, I can successfully push with no problem. I just added a bit more information to the question, with some excerpts from a trace of git 1.7.4.4 (successful push) vs a trace of git 1.8.1.2 (hangs). I don't know if that will help. I can't see any difference in what is being sent by the client to the server, apart from the User-Agent header. When using a git 1.7.4.4 client, the server responds to the second PROPFIND and continues, while when using a git 1.8.1.2 client, the server never responds. Puzzling! – D.W. Jun 15 '13 at 03:08
  • Hmm, okay. I'm lost as well. But yes, obviously the server does support DAV then :) – hobbs Jun 15 '13 at 03:26
  • Git 1.9 updated something related to HTTP 100 Continue, from the release notes: The HTTP transport, when talking GSS-Negotiate, uses "100 Continue" response to avoid having to rewind and resend a large payload, which may not be always doable. – onionjake Feb 20 '14 at 05:58
  • I'm thinking perhaps new git 1.9's behaviour is confusing the one you have locally? I'd try building latest version (install in your account) and trying again. – vonbrand Mar 11 '14 at 02:14
  • @D.W. Are you sure that the .netrc file is getting read? Perhaps it's an environmental issue (OS X vs. Linux). – John M. May 15 '14 at 07:05
  • I've downgraded to latest 1.8.x version: 1.8.5.2 but it's missing git-http-push which is required for WebDAV. Then I've dowgraded to 1.8.1.2 but I'm having exactly same problem. I have no idea what happened, this used to work just fine. – piotr_cz Aug 08 '14 at 18:02

1 Answers1

2

I've got problem with same symptoms (process hangs at HTTP/1.1 100 Continue).

I've found a commit that resolves the issue for me: http-push.c: make CURLOPT_IOCTLDATA a usable pointer, it's available in GIT release v2.0.3.

Apparently user is not being authenticated correctly on server even it does seems so from the communication.

Problem is weird, because for you it shows up on one machine but not on another with same GIT versions installed, for me GIT just stopped working within period of one month and I can't really say what was the cause.

Alternatively if it's possible, use GIT service provider that allows other protocols: SSH or HTTPS, these are much better debugged.

For windows users, here's my take on git-http-push 2.1.0-rc2 build (based on msysGit project instructions)

To backport to your msysGit installation, unzip it and in msysGit installation folder copy into/ overwrite libexec\git-core\git-http-push.exe.

The issue has been reported here.

Update

Latest msysGit (Git-1.9.4-preview20140815) contains backported bugfix.

piotr_cz
  • 8,755
  • 2
  • 30
  • 25