Incorrect message length when using C python api

Question

I'm trying to extend libhydrogen (open source found here: https://github.com/jedisct1/libhydrogen/) to support calls to the underlying library in Python. However, I ran into a strange issue. Specifically I have the following code:

static PyObject* crypto_secretbox_encrypt(PyObject* self, PyObject* args){
    char* m;
    size_t mlen;
    uint64_t msg_id;
    char *ctx;
    size_t ctx_len;
    const uint8_t *key; //this shouldn't matter its secret
    size_t key_len;
    PyArg_ParseTuple(args, "s#|K|s#|s#", &m, &mlen, &msg_id, &ctx, &ctx_len, &key, &key_len); //only s# accepts null bytes, need to random pointers to store lengths
    //printf("Parsed \n");
    //printf("MSG: %s, MSG_ID: %" PRIu64 ", CTX: %s\n", m, msg_id, ctx);
    if(ctx_len != hydro_secretbox_CONTEXTBYTES){
        PyErr_Format(PyExc_ValueError, "Context not of correct size: Received %lu bytes", ctx_len);
        return NULL;
    }

    if(key_len != hydro_secretbox_KEYBYTES){
        PyErr_Format(PyExc_ValueError, "Key not of correct size: Received %lu bytes", key_len);
        return NULL;
    }

Which is able to successfully parse the length of ctx, however, when it gets to the key it gets thrown the error,

  File "cryptotest.py", line 7, in <module>
    c1 = crypto.secretbox_encrypt("message1", 0, "\x00"*8, '\x00'*32);
ValueError: Key not of correct size: Received 140123308032032 bytes

Any ideas on why I'm unable to successfully parse the key length?

score 2 · Answer 1 · answered Feb 28 '18 at 00:28

2

You've specified the length as a size_t. The s# format does not take a size_t. It takes an int or Py_ssize_t, depending on whether the PY_SSIZE_T_CLEAN macro was #defined before including Python.h.

Note: For all # variants of formats (s#, y#, etc.), the type of the length argument (int or Py_ssize_t) is controlled by defining the macro PY_SSIZE_T_CLEAN before including Python.h. If the macro was defined, length is a Py_ssize_t rather than an int. This behavior will change in a future Python version to only support Py_ssize_t and drop int support. It is best to always define PY_SSIZE_T_CLEAN.

answered Feb 28 '18 at 00:28

user2357112

260,549
28
431
505

Thanks! I just realized that I was casting to the wrong data type and that was a potential issue, which I assume some type conversion messes up the value of number when it gets parsed. I can't find documentation on Py_ssize_t, but would I be able to just cast to size_t without an issue? – Hanson Duan Feb 28 '18 at 00:32
@HansonDuan: The closest thing to documentation for `Py_ssize_t` is [PEP 353](https://www.python.org/dev/peps/pep-0353/), which specifies it as the same size as `size_t` but signed, and as a typedef for `ssize_t` when `ssize_t` is available. – user2357112 Feb 28 '18 at 00:41

Incorrect message length when using C python api

1 Answers1