0

My task is it to write a script using opencv which will later run as a Celery task. What consequences does this have? What do I have to pay attention to? Is it enough in the end to include two lines of code or could it be, that I have to rewrite my whole script?

I read, that Celery is a "asynchronous task queue/job queuing system based on distributed message passing", but I wont pretend to know completely what that all entails.

I try to update the question, as soon as I get more details.

Framester
  • 33,341
  • 51
  • 130
  • 192
  • Care to share why you are downvoting or request closing? – Framester May 24 '12 at 10:13
  • 2
    I did not downvote this, but I guess it got downvoted for being too open. Same for the request of closing. – Niek de Klein May 24 '12 at 11:02
  • You didn't think you might find this information on the Celery website? – Marcin May 25 '12 at 10:21
  • He won't. The celery website is very comprehensive but completly unfriendly to newcomers. I remember my first days discovering celery, it was a documentation marathon nightmare. It's not it's lacking information, cause when you know what to look for, you find it. It's just that it assumes you know what you are doing. But queues are not part of any IT training that I know, and most web programmers never heard of it. – Bite code May 25 '12 at 10:54

1 Answers1

2
  • Celery implies a daemon using a broker (some data hub used to queue tasks). The celeryd daemon and the broker (RabbitMQ, redis, MongoDB or else) should always run in the background.
  • Your tasks will be queued, this means they won't happen all at the same time. You can choose how many at the same time can be run as a maximum. The rest of them will wait for the others to finish before starting. This also means some concurrency is often expected, and that you must create tasks that play nice with others doing the same thing.
  • Celery is not meant to run scripts but tasks, written as python functions. You can of course execute external scripts from Python, but your entry point is always a Python function.
  • Celery uses Kombu, which uses a message broker to dispatch the tasks. This implies the data you pass to your tasks should be serializable.
Bite code
  • 578,959
  • 113
  • 301
  • 329
  • Thanks for the answer. It contained a lot of things I have no idea about, so I will tell my boss, I will need some time to read into it. – Framester May 24 '12 at 17:43
  • 1
    You may want to ask a more precise question about any point above. Celery, and queuing in general, is not easy to grasp for newcomers if the only thing to explain you what's going on is internet. – Bite code May 24 '12 at 20:32