Limit HTTP POST upload bandwith

Question

In my application, I'm uploading continuously videoclips, worth of tens of megabytes of video to a server in the cloud.

The uploading part has been written in python aiohttp, while the receiving part in the cloud uses node.js and the "request" library.

I'd like to limit the bandwith of the HTTP upload, say, to 1Mbps or less.

Is this even possible within the HTTP protocol? Am I supposed to do this on the client (uploader) or the server (receiver) side?

Been doing a lot of googling, but the results deal mostly with "rate limiting" which is a different matter alltogether.

Help appreciated.

We need to see some code, but you are going to want to chunk the video coming in on the request side or python side. You get the original file size and divide by how much a second you want to upload. Then go from there. You will be doing this on the server end. — ABC, Oct 04 '19 at 08:48

ABC · Answer 1 · 2019-10-04T09:12:55.623

In response to my comment. You are chunking the upload POST or GET request and limiting it as you go, it is done on the server side. I would do the limiting on the upload side (not the response side), because that is what you are trying to control. So I would suggest this is a Python question since your upload handler is in Python.

Notice:

In the example, it says read a certain size of the payload when the upload form file is selected. You grab the size.
Then you limit based on the initial size.
You will be reading chunks, then will be returning where you left off.
Set a sleep handler, to chunk it at a certain speed.
Then callback the function with the left off chunk position and read position, to continue.

Limit the download GET speed, can be applied to POST also.

aiohttp Python example: https://github.com/aio-libs/aiohttp/issues/2638

import aiohttp
import asyncio


async def read(self, n: int=-1) -> bytes:
    """Read up to 'n' bytes of the response payload.

    If 'n' is -1 (default), read the entire payload.
    """
    if self._body is None:
        try:
            if n is -1:
                self._body = await self.content.read()
            else:
                chunks = []
                i = 0
                while i < n:
                    chunk = await self.content.read(n=n - i)
                    if not chunk:
                        break
                    chunks.append(chunk)
                    i += len(chunk)

                self._body = b''.join(chunks)

            for trace in self._traces:
                await trace.send_response_chunk_received(self._body)

        except BaseException:
            self.close()
            raise
    elif self._released:
        raise aiohttp.ClientConnectionError('Connection closed')

    return self._body


async def f(url, n=-1):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            content = await read(response, n=n)
            print(len(content))


URL = 'https://upload.wikimedia.org/wikipedia/commons/thumb/7/71/2010-kodiak-bear-1.jpg/320px-2010-kodiak-bear-1.jpg'

asyncio.run(f(URL, 10_000_000))  # 10 MB
asyncio.run(f(URL, 100))
asyncio.run(f(URL))

In that example of yours, you are GETting or POSTing something to the server. Then you have modded the response reader. I don't want to mod the response reader at all. I wan't to control the POSTing to the server, not the response reader to that POST. — El Sampsa, Nov 04 '19 at 14:45

Limit HTTP POST upload bandwith

1 Answers1