Generating HMAC signature with hashed "password"

Question

I am playing around with authentication methods in my RESTful API project and I really like the idea of generating HMAC-SHA256 signatures as an authentication method.

The client is creating the signature with few simple steps:

# example client-side code
sig = hmac.new(bytes('SUPER_SECRET_KEY', 'utf-8'), b'', sha256)
sig.update(request_path)
sig.update(request_body)
# ...other variables needed for generating the signature...

signature = sig.hexdigest()

and adding it to request header along with his "user name" (e.g. Authorization: THE_USER_NAME:abcd1234xyz890).

On server-side, I am trying to re-create it in the same way:

# example server-side code
def do_check(request):
    # get user name from request header
    username = request.headers['Authorization'].split(':')[0]

    # some method to retrieve the "secret key" from database
    user = db.User().filter(username=username).one()

    # use user's "secret key" to generate the signature
    sig = hmac.new(bytes(user.key, 'utf8'), b'', sha256)
    sig.update(bytes(request.path, 'utf-8'))
    sig.update(request.data)
    # ...other variables needed for generating the signature...

    return sig.hexdigest()
    # compare the returned signature with the one client sent us...

All this works fine as long as I store the user's key as a plain text in my database:

| username      | key              |
------------------------------------
| THE_USER_NAME | SUPER_SECRET_KEY |

We all are aware that this is absolutely unacceptable, so I tried to simply hash the SUPER_SECRET_KEY with bcrypt and storing a hashed string instead:

| username      | key                                                          |
--------------------------------------------------------------------------------
| THE_USER_NAME | $2b$12$UOIKEBFBedbcYFhbXBclJOZIEgSGaFmeZYhQtaE4l6WobFW1qvIf6 |

The problem I am facing now is that client used un-hashed version of the "secret key" to generate the signature which I am unable to do on server-side since I don't have it in plain-text anymore.

One of the examples of a similar approach is generating HMAC signature in Amazon Web Services (also simplified explanation of the same process) which does not require any additional log-ins or authentication, nor does provide any tokens or "replacements" for the key/secret combination. I really doubt that AWS is storing the secret in a plain text in database(?)

How can I recreate the HMAC signature on server-side with hashed version of "secret key" in database, while not forcing the client-side to change its method of signature generating (e.g. avoid installing bcrypt or even hashing the secret at all)?

Kenny Ostrom · Accepted Answer · 2017-07-25T18:07:53.530

1

Password hashing does not use a shared secret. The act of hashing the secret is supposed to destroy the actual value, while retaining the ability to authenticate a password. You can't reasonably be expected to recover the password from the hash.

Hmac authentication and validation uses a shared secret. Both parties must know this secret.

For this reason, password hashing is fundamentally different from hmac, and you can't simply hash the hmac key. The hash will not allow you to ever get back to the actual key.

[deleted irrelevant sections after clarification]

So you have to have some kind of secret somewhere, but it does not need to be in the database. The actual hmac shared secret can be encrypted in the database using a symmetric cipher (using a different key that is not in the database). Thus the server reads the encrypted hmac secret key, decrypts it, and uses that.

The important thing is you have to encrypt it in some way that you can decrypt, and that rules out hashing.

edited Jul 25 '17 at 18:07

answered Jul 24 '17 at 16:52

Kenny Ostrom

5,639
2
21
30

Thanks for clarifying some stuff here!! I tried to think about your sentence "...you typically authenticate some other way, then use hmac for the actual messages" but there are services which use the same approach as I described, even without additional authentication... I updated my answer with real world example, please check it out (the paragraph before the last one). – errata Jul 25 '17 at 10:29
You'll note that example is a shared secret. You create it by logging in to manage access keys (http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html), and then it's your responsibility to get it to the right people and protect it from the wrong people. – Kenny Ostrom Jul 25 '17 at 17:07
Yes, I understand that :) There is a "shared secret" in my example too (SUPER_SECRET_KEY)... What I am trying to figure out is how to avoid storing that key as an unhashed string in my database... – errata Jul 25 '17 at 17:13
Oh, you just don't want it "in the database" in plaintext, that's reasonable. You can encrypt it in the database with a symmetric encryption cipher or PKPK. Either way, you have to decrypt it by having someone type in some other secret, or having some other secret in plaintext somewhere ... just not in the database. The docs you linked mention a common solution of putting it in plaintext in a configuration file, and relying on the operating system to keep it secure behind someone's OS login. – Kenny Ostrom Jul 25 '17 at 17:27
Yeah, I am not worried at all how client-side is going to store it or hide it... I am worried of exposing the secret in my database. So, what you suggest is to use some hashing function in my database rather than using "varchar" or similar column type and storing it that way? Something similar to this? https://stackoverflow.com/questions/2647158/how-can-i-hash-passwords-in-postgresql – errata Jul 25 '17 at 17:40
What? No, hash is one way. You want a symmetric cipher https://www.google.com/search?q=symmetric+cipher&ie=utf-8&oe=utf-8 or really anything that you can decrypt. – Kenny Ostrom Jul 25 '17 at 17:59
Ah, alright, so if I understood properly, we talk about some sort of "double hashing" here? I should be able to decrypt the string from database on server-side with some secret only known to me in order to get a "real" secret which client was using too, is that right? – errata Jul 25 '17 at 18:41
I gave a link. "Hash" is the wrong word, but ... basically yes. – Kenny Ostrom Jul 25 '17 at 18:55
Alright! Thanks a lot for all the help, I think you deserved to get your answer accepted ;) – errata Jul 25 '17 at 18:56
As a bonus, the database may provide the encryption/decryption functions already. For example, see https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_aes-encrypt – Kenny Ostrom Jul 25 '17 at 18:59
I see... That's what I was referring to in my previous comment, I will definitely have a word with a database admin regarding it and we will come up with some nice solution. Thank you one more time!! – errata Jul 25 '17 at 19:01

Generating HMAC signature with hashed "password"

1 Answers1