0

I am trying to construct an AWS Signature v4 Auth header to call the STS GetCallerIdentity API as per the documentation at https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html in Bash.

Now, I have the same process working in Python, and after poring minutely over my scripts and outputs in Python and Bash, I see that the SHA256 calculated in Bash for the string is different from the one calculated in Python.

The strings look the same in my text editor - character by character.

But since the SHA256 differs, I am assuming that this much be a problem with encoding of the string.

The Python script uses UTF8, and even though I have tried doing a printf "%s" "${string}" | iconv -t utf-8 | openssl dgst -sha256 in the Bash script, the hash values still differ.

How do I convert bash strings/variables to UTF8 before calculating the SHA256 sum.

krish7919
  • 892
  • 2
  • 13
  • 30
  • 2
    Hex dump the input in both cases and find out how they differ – that other guy Feb 17 '19 at 05:50
  • Add your Python code to your question. – Cyrus Feb 17 '19 at 06:19
  • So I know the issue now @thatotherguy, thanks for the suggestion. The `\n` is interpreted properly in Python as `0x0A`, but shows up as `5C6E` in bash hexdump output. Does the `printf "%s" "${string}"` not handle it? – krish7919 Feb 17 '19 at 10:15
  • I feel ashamed calling myself an engineer and not thinking about dumping the byte string myself. I cannot believe I wasted so much time on this. :'( If you add your comment as answer I would accept it. – krish7919 Feb 17 '19 at 11:25

1 Answers1

4

It might helpful to see how you're calculating it in Python. From what I can see, it looks like the output is the same.

$ python -c "import hashlib; \
print(hashlib.sha256('test'.encode('utf8')).digest().hex())"
9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

$ printf "%s" test | openssl dgst -sha256
(stdin)= 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
Kevin Pielacki
  • 41
  • 1
  • I used `print(hashlib.sha256('test'.encode('utf8')).hexdigest())"` The bash one is exactly the same, except that no I do a `iconv` call in between, – krish7919 Feb 17 '19 at 09:20