What is the reason for _secret_backdoor_key variable in Python HMAC library source code?

Question

When I was browsing Python HMAC module source code today I found out that it contains global variable _secret_backdoor_key. This variable is then checked to interrupt object initialization.

The code looks like this

# A unique object passed by HMAC.copy() to the HMAC constructor, in order
# that the latter return very quickly.  HMAC("") in contrast is quite
# expensive.
_secret_backdoor_key = []

class HMAC:
    """RFC 2104 HMAC class.  Also complies with RFC 4231.

    This supports the API for Cryptographic Hash Functions (PEP 247).
    """
    blocksize = 64  # 512-bit HMAC; can be changed in subclasses.

    def __init__(self, key, msg = None, digestmod = None):
        """Create a new HMAC object.

        key:       key for the keyed hash object.
        msg:       Initial input for the hash, if provided.
        digestmod: A module supporting PEP 247.  *OR*
                   A hashlib constructor returning a new hash object.
                   Defaults to hashlib.md5.
        """

        if key is _secret_backdoor_key: # cheap
            return

Full code is here.

Does anyone know what is the reason for this variable? Comment says it is there for HMAC to return quicker than blank string (""). But why would user want to pass empty key to HMAC function? Is variable naming just a joke from HMAC developer or is it really some sort of backdoor?

Martijn Pieters · Accepted Answer · 2016-07-12T13:39:41.587

To create a copy of the HMAC instance, you need to create an empty instance first.

The _secret_backdoor_key object is used as a sentinel to exit __init__ early and not run through the rest of the __init__ functionality. The copy method then sets the instance attributes directly:

def copy(self):
    """Return a separate copy of this hashing object.

    An update to this copy won't affect the original object.
    """
    other = self.__class__(_secret_backdoor_key)
    other.digest_cons = self.digest_cons
    other.digest_size = self.digest_size
    other.inner = self.inner.copy()
    other.outer = self.outer.copy()
    return other

You could get the same effect with self.__class__('') (an empty string), but then HMAC.__init__ does a lot of unnecessary work as the attributes on the instance created are going to be replaced anyway. Note that using HMAC('') is a valid way to create an instance, you'd not want an instance devoid of any state in that case. By passing in the sentinel, HMAC.copy() can avoid all that extra work.

You could use a different 'flag' value, like False, but it is way too easy to pass that in because of a bug in your own code. You'd want to be notified of such bugs instead. By using a 'secret' internal sentinel object instead, you avoid such accidental cases.

Using [] as a sentinel unique object is quite an old practice. These days you'd use object() instead. The idea is that the sentinel is a unique, single object that you test against for identity with is. You can't re-create that object elsewhere, the is test only works if you pass in an reference to the exact same single object.

thanks for answer this is interesting, but why use some sentinel value rather than empty string? Why would empty string or boolean False be slower? — Pawel Miech, Jul 12 '16 at 09:14
@PawelMiech: passing in an empty string is a *valid use case* and you'd want the instance to be created properly. `False` would work, but you'd have to document that. This is for internal use only, and the sentinel value is just as internal and secret. You'd not accidentally pass in that object. — Martijn Pieters, Jul 12 '16 at 09:14
ok that explains it. But it is probably still possible to pass [] as accident or bug, isn't it? Wouldn't it be better to add explicit keyword argument that would stop init when it knows init is called from copy()? e..g extra argument for init:"early_exit" that would default to False and would be True if called from copy. — Pawel Miech, Jul 12 '16 at 09:20
@PawelMiech: no, if you pass in `[]` that'll be a **different list**, and `is` tests for *identity*. Note that the code tests for `key is _secret_backdoor_key`, not for `key == _secret_backdoor_key`. — Martijn Pieters, Jul 12 '16 at 09:21

What is the reason for _secret_backdoor_key variable in Python HMAC library source code?

1 Answers1