How to upload PDF file using request POST method and requests must be formatted as multipart MIME using python for xeroAPI?

Question

I'm trying to upload PDF-file in the Xero account using the python request library (POST method) and Xeros FilesAPI said "Requests must be formatted as multipart MIME" and have some required fields (link) but I don't how to do that exactly...If I do GET-request I'm getting the list of files in the Xero account but having a problem while posting the file (POST-request)...

My Code:

post_url = 'https://api.xero.com/files.xro/1.0/Files/'
files = {'file': open('/home/mobin/PycharmProjects/s3toxero/data/in_test_upload.pdf', 'rb')}

response = requests.post(
    post_url,
    headers={
        'Authorization': 'Bearer ' + new_tokens[0],
        'Xero-tenant-id': xero_tenant_id,
        'Accept': 'application/json',
        'Content-type': 'multipart/form-data; boundary=JLQPFBPUP0',
        'Content-Length': '1068',
    },
    files=files,
)

json_response = response.json()

print(f'Uploading Responsoe ==> {json_response}')
print(f'Uploading Responsoe ==> {response}')

Error Mesage/Response:

Uploading Responsoe ==> [{'type': 'Validation', 'title': 'Validation failure', 'detail': 'No file is was attached'}]
Uploading Responsoe ==> <Response [400]>

serghei · Accepted Answer · 2021-04-17T22:11:45.187

As I see you're improperly set the boundary. You set it in the headers but not tell to requests library to use custom boundary. Let me show you an example:

>>> import requests
>>> post_url = 'https://api.xero.com/files.xro/1.0/Files/'
>>> files = {'file': open('/tmp/test.txt', 'rb')}
>>> headers = {
...    'Authorization': 'Bearer secret',
...    'Xero-tenant-id': '42',
...    'Accept': 'application/json',
...    'Content-type': 'multipart/form-data; boundary=JLQPFBPUP0',
...    'Content-Length': '1068',
... }
>>> print(requests.Request('POST', post_url, files=files, headers=headers).prepare().body.decode('utf8'))
--f3e21ca5e554dd96430f07bb7a0d0e77
Content-Disposition: form-data; name="file"; filename="test.txt"


--f3e21ca5e554dd96430f07bb7a0d0e77--

As you can see the real boundary (f3e21ca5e554dd96430f07bb7a0d0e77) is different from what was passed in the header (JLQPFBPUP0).

You can actually directly use the requests module to controll boundary like this:

Let's prepare a test file:

$ touch /tmp/test.txt
$ echo 'Hello, World!' > /tmp/test.txt

Test it:

>>> import requests
>>> post_url = 'https://api.xero.com/files.xro/1.0/Files/'
>>> files = {'file': open('/tmp/test.txt', 'rb')}
>>> headers = {
...     'Authorization': 'Bearer secret',
...     'Xero-tenant-id': '42',
...     'Accept': 'application/json',
...     'Content-Length': '1068',
... }
>>> body, content_type = requests.models.RequestEncodingMixin._encode_files(files, {})
>>> headers['Content-type'] = content_type
>>> print(requests.Request('POST', post_url, data=body, headers=headers).prepare().body.decode('utf8'))
--db57d23ff5dee7dc8dbab418e4bcb6dc
Content-Disposition: form-data; name="file"; filename="test.txt"

Hello, World!

--db57d23ff5dee7dc8dbab418e4bcb6dc--

>>> headers['Content-type']
'multipart/form-data; boundary=db57d23ff5dee7dc8dbab418e4bcb6dc'

Here boundary is the same as in the header.

Another alternative is using requests-toolbelt; below example taken from this GitHub issue thread:

from requests_toolbelt import MultipartEncoder

fields = {
    # your multipart form fields
}

m = MultipartEncoder(fields, boundary='my_super_custom_header')
r = requests.post(url, headers={'Content-Type': m.content_type}, data=m.to_string())

But it is better not to pass bundary by hand at all and entrust this work to the requests library.

Update:

A minimal working example using Xero Files API and Python request:

from os.path import abspath
import requests

access_token = 'secret'
tenant_id = 'secret'

filename = abspath('./example.png')

post_url = 'https://api.xero.com/files.xro/1.0/Files'
files = {'filename': open(filename, 'rb')}
values = {'name': 'Xero'}

headers = {
    'Authorization': f'Bearer {access_token}',
    'Xero-tenant-id': f'{tenant_id}',
    'Accept': 'application/json',
}

response = requests.post(
    post_url,
    headers=headers,
    files=files,
    data=values
)

assert response.status_code == 201

You have put effort into answer but not work for me and not making sense with a question.. — Mobin Al Hassan, Apr 15 '21 at 10:39
What does "not work" mean? Could you describe the error a bit more and give an example of the output? Why do you think my answer didn't make sense? Could you elaborate on that? I'm sorry, but if you really want to figure it out, "doesn't work" is not enough. — serghei, Apr 15 '21 at 11:04
Please don't mind I really appreciate your effort...well actually the method you describe using that I'm still unable to upload PDF-file. — Mobin Al Hassan, Apr 15 '21 at 12:58
Well, as I said the issue is in improperly set the boundary. Request body and headers has different boundary. You can see it by debugging your code — serghei, Apr 15 '21 at 13:35
how to set same boundary, my boundry sloud be "JLQPFBPUP0" in both. — Mobin Al Hassan, Apr 16 '21 at 10:39
Actually, this is not working I'm getting , I'm trying to upload a PDF File — Mobin Al Hassan, Apr 16 '21 at 13:14
Could you provide the steps to reproduce or response example from the API server? — serghei, Apr 16 '21 at 22:51
https://github.com/mobinalhassan/StackoverflowQuestions/blob/main/s3_to_xero.py here is each line of code... — Mobin Al Hassan, Apr 17 '21 at 05:29
I provided a minimal working example using Xero Files API and Python request. See: https://github.com/mobinalhassan/StackoverflowQuestions/pull/1. And I said before -- the issue was in boundary. You should not set it by hand, or provide the same in the request body — serghei, Apr 17 '21 at 21:18

score 1 · Answer 2 · answered Apr 19 '21 at 23:12

I've tested this with Xero's Files API to upload a file called "helloworld.rtf" in the same directory as my main app.py file.

var1 = "Bearer "
var2 = YOUR_ACCESS_TOKEN
access_token_header = var1 + var2
body = open('helloworld.rtf', 'rb')

mp_encoder = MultipartEncoder(
fields={
     'helloworld.rtf': ('helloworld.rtf', body),
     }
 )

r = requests.post(
    'https://api.xero.com/files.xro/1.0/Files',
    data=mp_encoder,  # The MultipartEncoder is posted as data
    # The MultipartEncoder provides the content-type header with the boundary:
    headers={
        'Content-Type': mp_encoder.content_type,
        'xero-tenant-id': YOUR_XERO_TENANT_ID,
        'Authorization': access_token_header
    }
)

score 1 · Answer 3 · answered Apr 19 '21 at 23:20

looks like you got it solved. For reference and any future developers who are using the Xero supported package (https://github.com/XeroAPI/xero-python)

We just added the files_api example code to the sample app so the following would upload a file if you were using the Python SDK

https://github.com/XeroAPI/xero-python-oauth2-app/pull/29/files

name = "my-image"
filename= "my-image.jpg"
mime_type = "image/jpg"
with open('my-image.jpg', 'rb') as f:
    body = f.read()

try:
    file_object = files_api.upload_file(
        xero_tenant_id, 
        name = name, 
        filename= filename, 
        mime_type = mime_type,
        body=body
    )
except AccountingBadRequestException as exception:
    json = jsonify(exception.error_data)
else:
    json = serialize_model(file_object)

How to upload PDF file using request POST method and requests must be formatted as multipart MIME using python for xeroAPI?

3 Answers3