2

I have a web hook in my GitHub repo that is triggered every time there is a push event. Let's say the push event JSON looks like this as an example:

{
  "ref": "refs/heads/develop"
  "repository": {
    "id": 123456789,
    "name": "SweetSweetRepo"
  }
}

I provided a secret to the GitHub webhook and GitHub tells me in the request headers web hook UI that it created a SHA256 of abc123456 (for example).

I'm trying to create a proxy for the webhook but I cannot create this SHA on my own! I know my SHA function works because I tested it on example strings like 'hello world' and checked them vs. online converters. But I don't understand how I'm supposed to strip, escape, stringify, or otherwise format that GitHub JSON payload so it gives me the desired SHA.

UPDATE:

Here is a full example. I created a new repo. Here is a link to the web hook so you can see for yourself. https://github.com/fbomb111/webhook-test/settings/hooks/294937131

Here are the GitHub headers:

Request URL: http://example.com:
Request method: POST
Accept: */*
content-type: application/json
User-Agent: GitHub-Hookshot/5465ee1
X-GitHub-Delivery: 4ae55e0e-a9d6-11eb-87d7-8f44c61441ec
X-GitHub-Event: push
X-GitHub-Hook-ID: 294937131
X-GitHub-Hook-Installation-Target-ID: 363207517
X-GitHub-Hook-Installation-Target-Type: repository
X-Hub-Signature: sha1=9c62a9ab96bfe7a0f9b0b511dd9346a8f5ad7e69
X-Hub-Signature-256: sha256=aaaa0a8550aba58490572b65f998950a242ac61e20f2e295f1c839f58e6b3a23

Here is the payload:

{
  "ref": "refs/heads/main",
  "before": "0000000000000000000000000000000000000000",
  "after": "3f07cfffce2cf13559f33f11561dc8ec139a33a9",
  "repository": {
    "id": 363207517,
    "node_id": "MDEwOlJlcG9zaXRvcnkzNjMyMDc1MTc=",
    "name": "webhook-test",
    "full_name": "me/webhook-test",
    "private": false,
    "owner": {
      "name": "me",
      "email": "me@gmail.com",
      "login": "me",
      "id": 482183,
      "node_id": "MDQ6VXNlcjQ4MjE4Mw==",
      "avatar_url": "https://avatars.githubusercontent.com/u/482183?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/me",
      "html_url": "https://github.com/me",
      "followers_url": "https://api.github.com/users/me/followers",
      "following_url": "https://api.github.com/users/me/following{/other_user}",
      "gists_url": "https://api.github.com/users/me/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/me/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/me/subscriptions",
      "organizations_url": "https://api.github.com/users/me/orgs",
      "repos_url": "https://api.github.com/users/me/repos",
      "events_url": "https://api.github.com/users/me/events{/privacy}",
      "received_events_url": "https://api.github.com/users/me/received_events",
      "type": "User",
      "site_admin": false
    },
    "html_url": "https://github.com/me/webhook-test",
    "description": null,
    "fork": false,
    "url": "https://github.com/me/webhook-test",
    "forks_url": "https://api.github.com/repos/me/webhook-test/forks",
    "keys_url": "https://api.github.com/repos/me/webhook-test/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/me/webhook-test/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/me/webhook-test/teams",
    "hooks_url": "https://api.github.com/repos/me/webhook-test/hooks",
    "issue_events_url": "https://api.github.com/repos/me/webhook-test/issues/events{/number}",
    "events_url": "https://api.github.com/repos/me/webhook-test/events",
    "assignees_url": "https://api.github.com/repos/me/webhook-test/assignees{/user}",
    "branches_url": "https://api.github.com/repos/me/webhook-test/branches{/branch}",
    "tags_url": "https://api.github.com/repos/me/webhook-test/tags",
    "blobs_url": "https://api.github.com/repos/me/webhook-test/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/me/webhook-test/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/me/webhook-test/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/me/webhook-test/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/me/webhook-test/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/me/webhook-test/languages",
    "stargazers_url": "https://api.github.com/repos/me/webhook-test/stargazers",
    "contributors_url": "https://api.github.com/repos/me/webhook-test/contributors",
    "subscribers_url": "https://api.github.com/repos/me/webhook-test/subscribers",
    "subscription_url": "https://api.github.com/repos/me/webhook-test/subscription",
    "commits_url": "https://api.github.com/repos/me/webhook-test/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/me/webhook-test/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/me/webhook-test/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/me/webhook-test/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/me/webhook-test/contents/{+path}",
    "compare_url": "https://api.github.com/repos/me/webhook-test/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/me/webhook-test/merges",
    "archive_url": "https://api.github.com/repos/me/webhook-test/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/me/webhook-test/downloads",
    "issues_url": "https://api.github.com/repos/me/webhook-test/issues{/number}",
    "pulls_url": "https://api.github.com/repos/me/webhook-test/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/me/webhook-test/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/me/webhook-test/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/me/webhook-test/labels{/name}",
    "releases_url": "https://api.github.com/repos/me/webhook-test/releases{/id}",
    "deployments_url": "https://api.github.com/repos/me/webhook-test/deployments",
    "created_at": 1619802247,
    "updated_at": "2021-04-30T17:04:07Z",
    "pushed_at": 1619802340,
    "git_url": "git://github.com/me/webhook-test.git",
    "ssh_url": "git@github.com: me/webhook-test.git",
    "clone_url": "https://github.com/me/webhook-test.git",
    "svn_url": "https://github.com/me/webhook-test",
    "homepage": null,
    "size": 0,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": null,
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "disabled": false,
    "open_issues_count": 0,
    "license": null,
    "forks": 0,
    "open_issues": 0,
    "watchers": 0,
    "default_branch": "main",
    "stargazers": 0,
    "master_branch": "main"
  },
  "pusher": {
    "name": "me",
    "email": "me@gmail.com"
  },
  "sender": {
    "login": "me",
    "id": 482183,
    "node_id": "MDQ6VXNlcjQ4MjE4Mw==",
    "avatar_url": "https://avatars.githubusercontent.com/u/482183?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/me",
    "html_url": "https://github.com/me",
    "followers_url": "https://api.github.com/users/me/followers",
    "following_url": "https://api.github.com/users/me/following{/other_user}",
    "gists_url": "https://api.github.com/users/me/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/me/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/me/subscriptions",
    "organizations_url": "https://api.github.com/users/me/orgs",
    "repos_url": "https://api.github.com/users/me/repos",
    "events_url": "https://api.github.com/users/me/events{/privacy}",
    "received_events_url": "https://api.github.com/users/me/received_events",
    "type": "User",
    "site_admin": false
  },
  "created": true,
  "deleted": false,
  "forced": false,
  "base_ref": null,
  "compare": "https://github.com/me/webhook-test/commit/3f07cfffce2c",
  "commits": [
    {
      "id": "3f07cfffce2cf13559f33f11561dc8ec139a33a9",
      "tree_id": "782748b547b6e62d246487abfa7210775795d58f",
      "distinct": true,
      "message": "Create Readme.md",
      "timestamp": "2021-04-30T13:05:40-04:00",
      "url": "https://github.com/me/webhook-test/commit/3f07cfffce2cf13559f33f11561dc8ec139a33a9",
      "author": {
        "name": "me",
        "email": "me",
        "username": "me"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "username": "web-flow"
      },
      "added": [
        "Readme.md"
      ],
      "removed": [

      ],
      "modified": [

      ]
    }
  ],
  "head_commit": {
    "id": "3f07cfffce2cf13559f33f11561dc8ec139a33a9",
    "tree_id": "782748b547b6e62d246487abfa7210775795d58f",
    "distinct": true,
    "message": "Create Readme.md",
    "timestamp": "2021-04-30T13:05:40-04:00",
    "url": "https://github.com/me/webhook-test/commit/3f07cfffce2cf13559f33f11561dc8ec139a33a9",
    "author": {
      "name": "me",
      "email": "me",
      "username": "me"
    },
    "committer": {
      "name": "GitHub",
      "email": "noreply@github.com",
      "username": "web-flow"
    },
    "added": [
      "Readme.md"
    ],
    "removed": [

    ],
    "modified": [

    ]
  }
}

Here is the secret 1234567890

Note the sha value in the header: X-Hub-Signature-256: sha256=aaaa0a8550aba58490572b65f998950a242ac61e20f2e295f1c839f58e6b3a23

Now go to a online converter such as this https://dinochiesa.github.io/hmachash/index.html

Use the exact same payload and secret. You won't get the same sha256 value.

My guess is because GitHub is using something other than pretty-printed json to do the sha calculation. But I've tried many variations of what that formatting might be without luck.

Frankie
  • 11,508
  • 5
  • 53
  • 60
  • Do you mean the `X-Hub-Signature-256` documented in https://docs.github.com/en/developers/webhooks-and-events/webhook-events-and-payloads#delivery-headers? – jonrsharpe Apr 30 '21 at 15:12
  • @jonrsharpe yes I do. My mastery of Google already led me to that document. Note that it does not mention how the json needs to be formatted in order to generate the sha. If you copy the format used in their "Webhook payload example" on that page, you will not get a correct sha. – Frankie Apr 30 '21 at 15:22
  • I'm not sure what you mean by *"you will not get a correct sha"* - the [only example](https://docs.github.com/en/developers/webhooks-and-events/webhook-events-and-payloads#example-delivery) on the page that includes the values of the headers has a truncated body, so I don't know how you're determining that. And the payload is just bytes, that happen to represent JSON, it's unclear what formatting you're thinking of. GitHub publish a JS implementation, Octokit, maybe have a look at what that's doing. – jonrsharpe Apr 30 '21 at 15:27
  • @jonrsharpe You are correct that the payload resolves to bytes. And some of those bytes will represent newlines, whitespaces, etc. Referencing my question, I don't know if I'm supposed to strip, escape, etc the json payload before using it to calculate a sha. So yes, I am unclear what formatting I need, which is what I hope someone will answer for me. – Frankie Apr 30 '21 at 16:15
  • I wouldn't expect you to change _any_ of them, just to hash the actual payload. But presumably you've tried that and had an issue, could you give a [mre]. – jonrsharpe Apr 30 '21 at 16:36
  • @jonrsharpe I've updated the post with an example. Thanks for your assistance. – Frankie Apr 30 '21 at 17:14
  • We can't see the settings of your repository. But if you `JSON.stringify` that structure you get something that hashes to `aaaa0a8...` on that site. – jonrsharpe Apr 30 '21 at 17:54
  • @jonrsharpe I'm so close. What tool are you using to stringify it? Some online tools are giving different outputs. My application is C#/.NET core but I don't think I'm using the right implementation there either to stringify it. – Frankie Apr 30 '21 at 18:09
  • Just the console in my browser: `{"ref":"refs/heads/main","before":"0000000000000000000000000000000000000000","after":"3f07cfffce2cf13559f33f11561dc8ec139a33a9",...`. But fundamentally you need to hash what you're actually sending as the payload - as long as they're _consistent_ it doesn't matter precisely how it's formatted. – jonrsharpe Apr 30 '21 at 18:10
  • Yes that worked! I got the right sha with that format. You can post an answer if you'd like and I'll accept it. I should have mentioned the reason I have to do it this way is because I am proxying the GitHub web hook to another service. So I don't control the backend, I have to match this sha exactly or I am rejected by their service. – Frankie Apr 30 '21 at 18:28

0 Answers0