0

I'm setting workers count to 4 (in Pyrogram client) to download multiple files at the same time, It is creating all the temporary files at first, but downloading one file at a time.
I'm downloading profile photos actually.

Github issue:
https://github.com/pyrogram/pyrogram/issues/83

My problem is not related to python. in the python side, downloads are running in parallel and I've confirmed this. The problem is related to Pyrogram library.

my code:

photos = await generator_to_list2(client.get_chat_photos(chat_member.user.id))
photos: list[pyrogram.types.Photo]
tasks = []
for index, photo in enumerate(photos):
    tasks.append(asyncio.create_task(
        client.download_media(
            message=photo.file_id,
            file_name=f'{path}/files/photos/{index}.jpg'
        )
    ))

await asyncio.gather(*tasks)
print('fully done!')
EBRAHIM
  • 17
  • 5
  • Does this answer your question? [Multiple async calls blocking](https://stackoverflow.com/questions/43518504/multiple-async-calls-blocking) – ti7 Jul 18 '23 at 17:18
  • please include a [Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve), specifically add your imports and show how you're configuring the 4 workers – ti7 Jul 18 '23 at 17:25

2 Answers2

0

Don't await each task, but asyncio.gather() them!

await asyncio.gather(*tasks)
ti7
  • 16,375
  • 6
  • 40
  • 68
  • This is not answering my question. I'm new to python, so I'm waiting manually for each task. which is working fine. my question is about Pyrogram library and why it is downloading only one file at a time. – EBRAHIM Jul 18 '23 at 17:07
  • ah, they're _sequentially_ downloading because you're _sequentially_ awaiting them – ti7 Jul 18 '23 at 17:17
  • I have tested and they are running in parallel, even with my code. in the question, I stated that all the temporary files are created at first, but download s are sequential. the problem is related to Pyrogram library. Also I've tested the code you suggested, the problem is still there. – EBRAHIM Jul 18 '23 at 17:22
0

Finally I found the answer myself.

Changing workers count is not enough. We should also set max_concurrent_transmissions to a higher number than 1, when creating client.

client = Client(
    name="a name",
    api_id=api_id,
    api_hash=api_hash,
    workers=8,
    max_concurrent_transmissions=8)
EBRAHIM
  • 17
  • 5