1

I've created a photo-hosting website with Django that uses the Photologue app (which depends on Python Imaging Library). When users upload photos, multiple resized photos are generated for each (e.g. small/medium/large, thumbnails, etc). The trouble is that the resizing process eats up 100% CPU and makes it so that Django can barely handle simultaneous requests (an order of magnitude, or more, slower). Unless I am way off here, it seems I have to limit this CPU usage somehow.

I've tried using "cpulimit" command to limit CPU percentage use on the PID to see if that would even solve the problem, but it seemed to have no effect. Can I somehow limit CPU usage by PIL? Is there some approach I'm missing? My server is an Ubuntu EC2 micro-tier, so could of lack RAM/CPU power be an issue? Seems unlikely to be the issue, since this problem occurs with only two simultaneous users.

UPDATE: As per Andre's suggestion, I've upgraded to Django 1.4.1, which has made it so the dev server allows concurrent requests; however, it's still unbearably slow because PIL is hogging the CPU.

harry_sundown
  • 63
  • 7
  • 1
    you could use Threading to time slice it ... – Joran Beasley Sep 05 '12 at 05:56
  • How do you serve the Django App? runserver before 1.4 was single threaded, with 1.4 its multithreaded and should support handling another request..... – Andre Bossard Sep 05 '12 at 06:25
  • Wow, very good point. I am evidently running Django 1.3.0. I didn't realize that runserver was single-threaded pre-1.4. This should at least solve half the problem. I still don't know if PIL's use of 99% CPU is a problem or not, but I'll try this out now and see. – harry_sundown Sep 05 '12 at 18:40

1 Answers1

0

You can speed up things by using latest Pillow (generally 7x faster for resizing than original PIL) or even Pillow-SIMD (up to 30x faster)

homm
  • 2,102
  • 1
  • 16
  • 33