I am doing data migration which deals with images/videos and such being downloaded and then sent to dropbox by using its api.
I'm using python-django for the entire web app but I imagine this will take a lot of bandwidth and there might be lot of issues where a failure of one image being saved shouldn't stop the entire migration.
Thus, is celery a good idea? Or Twisted?
I'm a bit confused about how this would help me. What I've in mind is to spawn a server/thread for the process of dealing with a single image or a small set of images and thus being able to do it on multiple threads.