4

I am trying to upload large files using the Fetch API and I'm running into a problem when I post data larger than 128MB in chrome and 256MB in Firefox. My question is is there anyway to increase this maximum through a configuration in either chrome or firefox? Am I just doing it wrong? Is there a better alternative for posting large data asynchronously?

Here's a short example that shows the problem: https://jsfiddle.net/hspw4bzo

function performFetch() {
    const megabytes = document.getElementById( 'megabytes' ).value * 1;
    const largeString = (new Array(megabytes * 1024 * 1024 )).join("x");

    const options = {
      redirect: 'follow',
      method: 'POST',
      body: largeString
    };

    fetch( 'https://jsfiddle.net/', options ).then( () => {
      console.log( 'success' )
    } )
  }

When you hit the "Go" button it'll initiate a POST request with a body thats 128MB in size. In chrome this causes the frame to crash.

DiverseAndRemote.com
  • 19,314
  • 10
  • 61
  • 70

2 Answers2

7

I found that when posting a large amount of data the use of a Blob mitigates the out of memory error thrown by firefox and the crashing in chrome. I arrived at Blob usage after viewing other answers here and here

  function performFetch() {
    const megabytes = document.getElementById( 'megabytes' ).value * 1;
    const largeString = (new Array(megabytes * 1024 * 1024 )).join("x");

    const options = {
      redirect: 'follow',
      method: 'POST',
      body: new Blob( [ largeString ], { type: 'text/plain' } )
    };

    fetch( 'http://example.com', options ).then( () => {
      console.log( 'success' )
    } )
  }
DiverseAndRemote.com
  • 19,314
  • 10
  • 61
  • 70
  • 2
    Isn’t that exactly repeating my answer? That’s what is proposed there: use blob instead of string. This is weird, asking a question and avoiding giving the credits to ones who solved your problem. – smnbbrv Aug 21 '18 at 17:53
  • Nope, not the same @smnbbrv. You use `FormData` with blob being used for a field in a multipart form post which was not in my question. My answer illustrates that it can be applied to the entire post body which in my case is a large JSON document which can include the contents of a very large file. I felt the answer I derived from the sited sources better suits the question I asked – DiverseAndRemote.com Aug 22 '18 at 01:50
6

You should not upload files as strings; that's also valid for the old good XMLHttpRequest. You would either come to the limits of the server or a browser (that one you are currently facing).

Use multipart upload of a Blob instead, e. g. like they do here:

const formData = new FormData()
formData.append('blob', new Blob(['Hello World!\n']), 'test')

fetch('http://localhost:5001/api/v0/add', {
  method: 'POST',
  body: formData
})
.then(r => r.json())
.then(data => {
  console.log(data)
})
smnbbrv
  • 23,502
  • 9
  • 78
  • 109