Why are the validations of signatures different?

Question

I need to validate a signature. The code examples for signature validation are only in Python. Here is the Python function:

import hmac
import hashlib
import json


def verify_signature(key, timestamp, provided_signature, payload):
  key_bytes = bytes.fromhex(key)
  payload_str = json.dumps(payload)
  data = timestamp + payload_str
  signature = hmac.new(key_bytes, data.encode('utf-8'),
                       hashlib.sha256).hexdigest()
  valid = hmac.compare_digest(provided_signature, signature)
  return valid

I need to translate some Python code into TypeScript.

Here is my best attempt:

export function verifyCloseSignature(
  request: Request,
  key: string,
  payload: any,
) {
  const headers = request.headers;

  const timestamp = headers.get('close-sig-timestamp');
  const providedSignature = headers.get('close-sig-hash');

  if (!timestamp) {
    throw new Error('[verifyCloseSignature] Required timestamp header missing');
  }

  if (!providedSignature) {
    throw new Error('[verifyCloseSignature] Required signature header missing');
  }

  const payloadString = JSON.stringify(payload);
  const hmac = crypto.createHmac('sha256', Buffer.from(key, 'hex'));
  hmac.update(timestamp + payloadString);
  const calculatedSignature = hmac.digest('hex');

  return crypto.timingSafeEqual(
    Buffer.from(providedSignature, 'hex'),
    Buffer.from(calculatedSignature, 'hex'),
  );
}

Why is this code not equivalent? Data that is validated by the Python code, fails the validation in JavaScript, when used like this:

const headers = new Headers();
headers.set('close-sig-hash', signature);
headers.set('close-sig-timestamp', timestamp.toString());
headers.set('Content-Type', 'application/json');
const request = new Request(faker.internet.url(), {
  method: 'POST',
  headers: headers,
  body: JSON.stringify(payload),
});

const actual = verifyCloseSignature(request, key, payload);
const expected = true;

expect(actual).toEqual(expected);

I expect the function to return true.

I'd start with checking if `json.dumps(payload)` produces the same string as `JSON.stringify(payload)` — Marat, Jun 15 '23 at 17:19
`JSON.stringify` returns a "compressed" JSON, without spaces after comma and colon. `json.dumps` adds spaces by default. You can change this behaviour in python, but I'm not aware of same ability in JS. — STerliakov, Jun 15 '23 at 17:34
https://stackoverflow.com/questions/24834812/space-in-between-json-stringify-output — STerliakov, Jun 15 '23 at 17:35

score 0 · Answer 1 · answered Jun 15 '23 at 17:37

The conversion from Python to TypeScript seems to have a little issue, otherwise it is fine.

In your Python snippet you use json.dumps() to convert the payload to a string value. This does not only convert the payload to a string, however it also sorts the keys in the payload.

This is important since when you use hmac.update() the order of the keys is very important. This means if the keys are not ordered exactly the same way when you verify the signature in TypeScript it will not match the original signature and the validation fails.

However you can fix that pretty fast by using JSON.stringify() since this doesn't sort the keys in TypeScript in the payload.

Here is a way how you could fix your TypeScript Code:

Code:

export function verifyCloseSignature(
  request: Request,
  key: string,
  payload: any
) {
  const headers = request.headers;

  const timestamp = headers.get("close-sig-timestamp");
  const providedSignature = headers.get("close-sig-hash");

  if (!timestamp) {
    throw new Error("[verifyCloseSignature] Required timestamp header missing");
  }

  if (!providedSignature) {
    throw new Error("[verifyCloseSignature] Required signature header missing");
  }

  const sortedPayload = JSON.stringify(payload, Object.keys(payload).sort());
  const hmac = crypto.createHmac("sha256", Buffer.from(key, "hex"));
  hmac.update(timestamp + sortedPayload);
  const calculatedSignature = hmac.digest("hex");

  return crypto.timingSafeEqual(
    Buffer.from(providedSignature, "hex"),
    Buffer.from(calculatedSignature, "hex")
  );
}

score 0 · Accepted Answer · answered Jun 19 '23 at 08:47

@SUTerliakov's SO link helped me to find a good solution. It is indeed caused by the extra white space.

Here is the code that I ended up with:

import { pipe, replace } from Ramda;

export const toJSONWithSpaces = pipe(
  (object: unknown) => JSON.stringify(object, null, 1), // stringify with line-breaks and indents
  replace(/\n +/gm, ' '), // replace line breaks and following spaces with a single space
  replace(/:\s/g, ': '), // ensure a space after colon
  replace(/{\s/g, '{'), // remove space after opening brace
  replace(/\s}/g, '}'), // remove space before closing brace
  replace(/\[\s/g, '['), // remove space after opening bracket
  replace(/\s]/g, ']'), // remove space before closing bracket
  replace(/,\s/g, ', '), // ensure a space after comma
);

export function verifyCloseSignature(
  request: Request,
  key: string,
  payload: any,
) {
  const headers = request.headers;

  const timestamp = headers.get('close-sig-timestamp');
  const providedSignature = headers.get('close-sig-hash');

  if (!timestamp) {
    throw new Error('[verifyCloseSignature] Required timestamp header missing');
  }

  if (!providedSignature) {
    throw new Error('[verifyCloseSignature] Required signature header missing');
  }

  const hmac = crypto.createHmac('sha256', Buffer.from(key, 'hex'));
  const cleanedPayload = toJSONWithSpaces(payload);
  hmac.update(timestamp + cleanedPayload);
  const calculatedSignature = hmac.digest('hex');

  return crypto.timingSafeEqual(
    Buffer.from(providedSignature, 'hex'),
    Buffer.from(calculatedSignature, 'hex'),
  );
}

Why are the validations of signatures different?

2 Answers2