Generating a SHA1 Signature with Different Results Between Ubuntu/Bash and Windows/Powershell

Question

Why would these respective bash and powershell scripts return different signatures:

## (Ubuntu) Bash
now='Fri, 26 Jul 2019 12:32:36 -0400'
bucket="mybucket"
path="/"
awsKey="MSB0M3K06ELMOI65QXI1"
awsSsecret="706Fdj+LFKf8pf/2Wh5V8Q8jbgGUUQo3xSXr5sbt"

## Create the signature
message="GET\n\n\n${now}\n/${bucket}${path}"
messageSignature=$(echo -en "${message}" | openssl sha1 -hmac "${awsSsecret}" -binary | openssl base64)
echo $messageSignature

Returns >> 5oJM2L06LeTcbYSfN3fDFZ9yt5k=

## Powershell
$OutputEncoding = [Text.UTF8Encoding]::UTF8

$now='Fri, 26 Jul 2019 12:32:36 -0400'
$bucket="mybucket"
$path="/"
$awsKey="MSB0M3K06ELMOI65QXI1"
$awsSsecret="706Fdj+LFKf8pf/2Wh5V8Q8jbgGUUQo3xSXr5sbt"

## Create the signature
$message="GET\n\n\n${now}\n/${bucket}${path}"
$messageSignature=$(echo -en "${message}" | openssl sha1 -hmac "${awsSsecret}" -binary | openssl base64)
echo $messageSignature

Returns >> 77u/W8O5RwJeBsOodDddXeKUlCQD4pSUduKVrD3ilIxADQo=

enter image description here ]2

On Ubuntu, my shell is running "en_US.UTF-8".

I've run into the case where the signature is different on different systems: AIX, Windows w/Ubuntu, Windows w/Powershell, etc. I'm trying to figure out why.

Command `echo` isn't standardized, you could try `printf '%s' "$string_to_sign"` — Lorinczy Zsigmond, Jul 26 '19 at 17:59
@LorinczyZsigmond That's great advice. In this scenario, the use of `echo` vs. `printf` makes no difference. However, when considering Ubuntu vs. AIX, it might, and I'll take that into consideration. Thank you. — Adam, Jul 27 '19 at 02:58

Gordon Davisson · Accepted Answer · 2019-07-27T03:18:20.200

There are at least three problems here: Powershell pipelines aren't binary-safe, you have the wrong variable name, and echo isn't portable/reliable.

At least according to this question, you can't pipe binary data in Powershell. The output of openssl (which is raw binary data) is getting treated as UTF-8 text (presumably due to $OutputEncoding), and mangled in the process. You can tell by decoding from base64 and looking in hex instead:
```
$ echo '77u/W8O5RwJeBsOodDddXeKUlCQD4pSUduKVrD3ilIxADQo=' | base64 -D | xxd
00000000: efbb bf5b c3b9 4702 5e06 c3a8 7437 5d5d  ...[..G.^...t7]]
00000010: e294 9424 03e2 9494 76e2 95ac 3de2 948c  ...$....v...=...
00000020: 400d 0a                                  @..
```
It starts with EF BB BF, which is a UTF-8 byte order mark; and it ends with 0D 0A, which is a DOS/Windows line ending (ASCII carriage return and linefeed characters). Something else bad is happening as well, since it's much too long for a sha1 hash, even if you account for the BOM and line ending.

The output of echo is probably getting mangled similarly, so even if the hash wasn't mangled it'd be the hash of the wrong byte sequence.

See this question (and its answer) for an example of using Powershell's own tools to compute the HMAC-SHA1 of a string.
The message to be signed is in $message, but you actually sign $string_to_sign, which is undefined. The Ubuntu result is the correct HMAC-SHA1 for the null string:
```
$ </dev/null openssl sha1 -hmac "706Fdj+LFKf8pf/2Wh5V8Q8jbgGUUQo3xSXr5sbt" -binary | openssl base64
NCRRWG4nL9sN8QMrdmCPmUvNlYA=
```
As Lorinczy Zsigmond pointed out, echo -en isn't predictable. Under some implementations/OSes/shells/compile & runtime options/phases of moon/etc, it might or might not print "-en" as part of its output, and maybe also print a newline (or something) at the end. Use printf instead:
```
printf 'GET\n\n\n%s\n/%s%s' "${now}" "${bucket}" "${path}" | openssl sha1 ...
```
Or (in bash, but not all other shells):
```
printf -v message 'GET\n\n\n%s\n/%s%s' "${now}" "${bucket}" "${path}"
printf '%s' "$message" | openssl sha1 ...
```
Or (bash again):
```
nl=$'\n'
message="GET${nl}${nl}${nl}${now}${nl}/${bucket}${path}"
printf '%s' "$message" | openssl sha1 ...
```

For people reviewing this answer to solve their own problem, the culprit was `...you can't pipe binary data in Powershell...` Items 1 and 3 were immaterial with regard to the given use-case. — Adam, Jul 27 '19 at 03:01
@Adam Yeah, the one I had as #2 really should've been #1. So I'm swapping them to make it more directly relevant for later readers. — Gordon Davisson, Jul 27 '19 at 03:16

Adam · Answer 2 · 2019-07-27T11:05:42.830

Providing a summary for anyone who stumbled onto this thread. The problem problems were....

Cannot pass binary data through the pipeline as noted by Gordon Davisson
Need to represent newlines with "`n" instead of "\n"

The Powershell script that ultimately replicated my results in Ubuntu/Bash was...

$now='Fri, 26 Jul 2019 12:32:36 -0400'
$bucket="mybucket"
$path="/"
$awsKey="MSB0M3K06ELMOI65QXI1"
$awsSsecret="706Fdj+LFKf8pf/2Wh5V8Q8jbgGUUQo3xSXr5sbt"
$message="GET`n`n`n${now}`n/${bucket}${path}"

## Create the signature
$sha = [System.Security.Cryptography.KeyedHashAlgorithm]::Create("HMACSHA1")
$sha.Key = [System.Text.Encoding]::UTF8.Getbytes($awsSsecret)
[Convert]::Tobase64String($sha.ComputeHash([System.Text.Encoding]::UTF8.Getbytes($message)))

While it's a better idea to use printf over echo on Linux/Unix systems, that difference wasn't material in this use-case.

Generating a SHA1 Signature with Different Results Between Ubuntu/Bash and Windows/Powershell

2 Answers2