0

Final result:

My apologies for the mistake.
I set wrong value to Content-Range in my http requests.

Example is here:

  1. Original file size: 5242880
  2. Chunk size: 2097152
  3. Required HTTP Request count: 3

Thus, the correct Content-Range is following these:

  1. bytes 0-2097151/5242880
  2. bytes 2097152-4194303/5242880
  3. bytes 4194304-5242879/5242880

Thanks to your advice, i got to test with:

  • cURL
  • Postman
  • Charles
  • Node.js
  • C#
  • Swift

Second try with Postman:

I tried again a following API with Postman.

I use rst_basic_sample_project.rvt that provide by Autdesk.
The file is 5693440 bytes.

$ du -b rst_basic_sample_project.rvt
5693440 rst_basic_sample_project.rvt

I divided with the following command.

# Chunk size is 5 MB.
split -b 5242880 rst_basic_sample_project.rvt "temp/data-"

So, i got data-aa and data-ab.
Next, try to use PUT buckets/:bucketKey/objects/:objectName/resumable.

Send data-aa to API. HTTP Request is here:

PUT /oss/v2/buckets/${MY-BUCKET-KEY}/objects/${MY-OBJECT-NAME}/resumable HTTP/1.1
Host: developer.api.autodesk.com
Authorization: Bearer ${MY-TOKEN}
Content-Range: bytes 1-5242880/5693440
session-id: j45e05
Content-Type: application/octet-stream

"<data-aa binary>"

I got HTTP 202. It's expected result.
And, send data-ab to API. HTTP Request is here:

PUT /oss/v2/buckets/${MY-BUCKET-KEY}/objects/${MY-OBJECT-NAME}/resumable HTTP/1.1
Host: developer.api.autodesk.com
Authorization: Bearer ${MY-TOKEN}
Content-Range: bytes 5242880-5693440/5693440
Content-Type: application/stream
session-id: j45e05
Content-Type: application/octet-stream

"<data-ab binary>"

I got HTTP 416 Requested Range Not Satisfiable.
It's unexpected result.

First try with cURL:

Please show me how to upload a large file.
I'm trying to upload a file larger than 100MB using the API below.

First of all, i simply want to know the API response and verify. Therefore, I sent HTTP PUT Request with cURL.

  • The large file is 157286400 bytes (Using command dd if=/dev/random of=bigfile bs=1M count=150 )
  • Data chunk size is 5242880 bytes (Using command split -b 5242880 bigfile "temp/data-" )

cURL command is here:

curl \
    -v \
    -X PUT "https://developer.api.autodesk.com/oss/v2/buckets/${MY-BUCKET-KEY}/objects/${MY-OBJECT-NAME}/resumable" \
    -H "Authorization:Bearer ${MY-TOKEN}" \
    -H "Content-Length: 5242880" \
    -H "Content-Type:text/plain; charset=UTF-8" \
    -H "Content-Range:bytes 1-5242880‬/157286400 " \
    -H "Session-Id:111111111" \
    --data @temp/data-aa # One of the files created by splitting a large file with the split command.

# PUT /oss/v2/buckets/${MY-BUCKET-KEY}/objects/${MY-OBJECT-NAME}/resumable HTTP/1.1
# Host: developer.api.autodesk.com
# User-Agent: curl/7.69.0
# Accept: */*
# Authorization:Bearer ${MY-TOKEN}
# Content-Length: 5242880
# Content-Type:text/plain; charset=UTF-8
# Content-Range:bytes 1-5242880‬/157286400 
# Session-Id:111111111

API response return HTTP/1.1 504 GATEWAY_TIMEOUT.
How to upload a large file than 100MB via Data Management API on the Autodesk Forge?

hiroakit
  • 57
  • 1
  • 6

2 Answers2

0

Try buckets/:bucketKey/objects/:objectName/resumable

You may then split larger files into chunks and specify a range in the header for each request to upload the chunks to avoid timeouts and other networking issues, e.g.:

curl -v "https://developer.api.autodesk.com/oss/v2/buckets/bucketexamplekey/objects/objectKeyFoo/resumable"
  -X PUT
  -H "Authorization:Bearer p0vNqwggSc4EOJdhR4KkTVyJFIzQ"
  -H "Content-Type:text/plain; charset=UTF-8"
  -H "Content-Range:bytes 1-15/100"
  -H "Session-Id:-811577637"
  --data ...

You might also want to use this tool here (guide here) for this purpose.

Bryan Huang
  • 5,247
  • 2
  • 15
  • 20
  • I tried to upload on https://forge-tools-hub.now.sh/uploader. I set chunk size 2 MB. Upload failed with the following pattern: - Test data 1: dd if=/dev/random of=bigfile bs=1M count=150 - Test data 2: [rst_basic_sample_project.rvt](https://developer.static.autodesk.com/forgeunified/releases/samples/rst_basic_sample_project.rvt) Can you upload in your environment? – hiroakit Apr 09 '20 at 08:28
  • [The uploader](https://forge-tools-hub.now.sh/uploader) doesn't clearly working. Talking on [GitHub - Failed to upload file](https://github.com/dukedhx/forge-tools-hub/issues/3) – hiroakit Apr 10 '20 at 04:35
  • Update my [question](https://stackoverflow.com/q/61097888/13257344). – hiroakit Apr 10 '20 at 06:55
0

If you can use a tool, and don't need to implement the resumable upload yourself (and you can use VSCode), you should check out VS-Code Forge Tools.

https://github.com/petrbroz/vscode-forge-tools

https://forge.autodesk.com/blog/forge-visual-studio-code

Our team has used it to upload GB sized models during testing.

mossherder
  • 135
  • 7