0

I am trying to write a simple python code to upload a file to Azure PageBlob. I am not sure what should I specify for the x-ms-blob-content-length as I keep getting error. The documentation is not very clear.

My code tries to left pad the file with 0s to make sure its within 512 byte boundary but I don't know if I am on the right path. Thank you.

import sys
import os
import http.client
from urllib.parse import urlparse

sas_uri = '<SAS URI here>'
uri = urlparse(sas_uri)

conn = http.client.HTTPSConnection(uri.hostname, port=uri.port, timeout=3000)

file_path = r"C:\Users\user\Downloads\npp.Installer.exe"

with open(file_path, 'rb') as reader:
    file = reader.read()

    size = os.stat(file_path).st_size
    boundary = size % 512
    if boundary != 0:
        file = file.ljust(boundary, b'\0')
        size = size + boundary
    headers = {
        'Content-Type': 'application/octet-stream',
        'Content-Length': 0,
        'x-ms-blob-type': 'PageBlob',
        'x-ms-blob-content-length': size
    }

    conn.request('PUT', sas_uri, file, headers)

    res = conn.getresponse()
    data = res.read()
    print(data.decode("utf-8"))

Error:

Connected to pydev debugger (build 211.7142.13)
<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidHeaderValue</Code><Message>The value for one of the HTTP headers is not in the correct format.
RequestId:44c84519-501c-0004-5ecc-937c22000000
Time:2021-08-18T00:57:43.5973565Z</Message><HeaderName>x-ms-blob-content-length</HeaderName><HeaderValue>3991344</HeaderValue></Error>
Node.JS
  • 1,042
  • 6
  • 44
  • 114

1 Answers1

1

Essentially the issue is with the following line of code:

size = size + boundary

If you divide this number by 512, you will notice that the number is not exactly divisible by 512.

In order to set the content length of the page blob to be a multiple of 512, you will need to use the following logic:

size = size + 512 - boundary

Please give that a try. It should work.

Gaurav Mantri
  • 128,066
  • 12
  • 206
  • 241
  • It worked. Thank you so much. I have another question, does this route actually uploads the file? because it runs so fast and I don't think it actually upload anything or maybe it just allocates space in blob? – Node.JS Aug 18 '21 at 02:31
  • You're correct. It simply creates an empty page blob with that size. You will need to call [`Put Page`](https://learn.microsoft.com/en-us/rest/api/storageservices/put-page) operation to populate the content in that blob. This is what the documentation says - `A call to a Put Blob to create a page blob or an append blob only initializes the blob. To add content to a page blob, call the Put Page operation. To add content to an append blob, call the Append Block operation.` – Gaurav Mantri Aug 18 '21 at 02:38
  • BTW, padding zero bytes so that a blob can be uploaded as a page blob is a bad idea. Essentially your file will be unusable. Is there a reason you're uploading this exe as a page blob? – Gaurav Mantri Aug 18 '21 at 02:39
  • Do I need to pad it with 0? I thought I need it to pad it with 0 to be within 512 byte boundary. We only have access to PageBlob and I am trying to use it as BlockBlob – Node.JS Aug 18 '21 at 02:42
  • `Do I need to pad it with 0? I thought I need it to pad it with 0 to be within 512 byte boundary.` - That's correct. `We only have access to PageBlob and I am trying to use it as BlockBlob` - I am very surprised to hear that. Is there a reason for that? Try to download this file you just uploaded as page blob and execute it. You should get a message that the file is corrupted. – Gaurav Mantri Aug 18 '21 at 02:48
  • So I shouldn't need to pad it with 0, makes sense. Thank you – Node.JS Aug 18 '21 at 02:50
  • If you do that, you won’t be able to save that as page blob. – Gaurav Mantri Aug 18 '21 at 02:54
  • Thank you so much for the help. I asked a follow-up question: https://stackoverflow.com/questions/68826122/pageblob-upload-as-one-chunk-x-ms-range-invalid – Node.JS Aug 18 '21 at 03:43