Why do I get a TypeError when passing a bytes string to magic.from_buffer()?

Question

I am trying to get the filetype of an image from the url using python-magic's from_buffer, and it works well in practice, but fails when I try to run a unittest.

This is my function:

def get_uri_from_url(url):
    if url != '':
        response = requests.get(url)
        data_type = magic.from_buffer(response.content, mime=True)
        image_bytes = str(base64.b64encode(response.content).decode('utf-8'))
        return f'data:{data_type};base64,{image_bytes}'
    return url

And this is the test so far:

    def test_get_uri_from_url(self):
        test_bytes = b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x05\x00\x00\x00\x05\x08\x06\x00\x00\x00\x8do&' \
                     b'\xe5\x00\x00\x00\x1cIDAT\x08\xd7c\xf8\xff\xff?\xc3\x7f\x06 \x05\xc3 \x12\x84\xd01\xf1\x82X\xcd' \
                     b'\x04\x00\x0e\xf55\xcb\xd1\x8e\x0e\x1f\x00\x00\x00\x00IEND\xaeB`\x82'

        class Requests:
            content = test_bytes

            def get(self, url):
                return self

        with patch('path_to_my_function.requests', return_value=Requests):
            self.assertEqual(
                get_uri_from_url('any_url'),
                f'data:image/png;base64,{test_bytes}'
            )

This test takes ages to run, and finally errors with the following traceback:

Traceback (most recent call last):
  File "/<path>/test_file.py", line 90, in test_get_uri_from_url
    get_uri_from_url('any_url'),
  File "/<path>/file.py", line 165, in get_uri_from_url
    data_type = magic.from_buffer(response.content, mime=True)
  File "/<path_to_python>/python3.6/site-packages/magic.py", line 148, in from_buffer
    return m.from_buffer(buffer)
  File "/<path_to_python>/python3.6/site-packages/magic.py", line 80, in from_buffer
    return maybe_decode(magic_buffer(self.cookie, buf))
  File "/<path_to_python>/python3.6/site-packages/magic.py", line 255, in magic_buffer
    return _magic_buffer(cookie, buf, len(buf))
ctypes.ArgumentError: argument 2: <class 'TypeError'>: wrong type

I can't see what I'm doing wrong, when I run this in a terminal it works fine, I must be missing something.

In [17]: response = requests.get('https://upload.wikimedia.org/wikipedia/commons/3/31/Red-dot-5px.png')                                                                           

In [18]: response.content                                                                                                                                                         
Out[18]: b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x05\x00\x00\x00\x05\x08\x06\x00\x00\x00\x8do&\xe5\x00\x00\x00\x1cIDAT\x08\xd7c\xf8\xff\xff?\xc3\x7f\x06 \x05\xc3 \x12\x84\xd01\xf1\x82X\xcd\x04\x00\x0e\xf55\xcb\xd1\x8e\x0e\x1f\x00\x00\x00\x00IEND\xaeB`\x82'

In [19]: magic.from_buffer(response.content, mime=True)                                                                                                                           
Out[19]: 'image/png'

Could you include the full traceback instead of just the last section? — glibdud, May 01 '19 at 12:36

score 0 · Answer 1 · answered May 01 '19 at 12:53

I have managed to make it work by using the following:

    def test_get_uri_from_url(self):

        class Response:
            content = b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x05\x00\x00\x00\x05\x08\x06\x00\x00\x00\x8do&' \
                      b'\xe5\x00\x00\x00\x1cIDAT\x08\xd7c\xf8\xff\xff?\xc3\x7f\x06 \x05\xc3 \x12\x84\xd01\xf1\x82X' \
                      b'\xcd\x04\x00\x0e\xf55\xcb\xd1\x8e\x0e\x1f\x00\x00\x00\x00IEND\xaeB`\x82'

        with patch('touchsurgery.apps.apiv3.helpers.requests.get', return_value=Response):
            self.assertEqual(
                get_uri_from_url('any_url'),
                'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHx'
                'gljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=='
            )

I believe the problem was in my mocking and patching. Since I patched the return value of 'requests', but requests itself is never explicitly called (), the object being passed to magic.from_buffer was just a magicmock object so it failed.

This function now works because get() is explicitly called in the function. If anyone else can shed some light on this or give a more detailed explanation it would be greatly appreciated.

From `from_buffer` [documentation](https://github.com/ahupp/python-magic/blob/a9b5592b6193a78530a312ea115820702b69746a/magic/__init__.py#L182) it _Accepts a binary string and returns the detected filetype. [...]_. That is what you're using as argument in this answer. It is a late response but might help someone else. — wladimirguerra, Oct 31 '21 at 20:35

Why do I get a TypeError when passing a bytes string to magic.from_buffer()?

1 Answers1