0

I work on a website who is using cron jobs for import and update products. The platform is Woocommerce. The supplier offered the csv file, and a server requirements. I have them over the requested: 32GB RAM, 16vCore, 400GB space.

The issue is that the job is working just few minutes/hour...between 3 and 10 minutes, even he have alot of pending jobs. I mean, for example, at from 09:00 to 09:10, the job is working, then is stopping, and he start again at 10:08 and is working until 10:15. Then from 11:10 to 11:20. I don't understand why. This is the cron job settings:

Minute Hour Day Month Weekday

* * * * * /usr/bin/flock -n /tmp/wc_process_import_itakeit.ro.lockfile /usr/bin/curl https://itakeit.ro/wcmipconnector/readfiles >> /home/itakeit/public_html/wp-content/uploads/mip-connector/logs/process_import_cron.log

I don't know what to put/change...in order to finish all pending processes.

I have another cron job who is working perfectly, and this is the syntax from another supplier:

Minute Hour Day Month Weekday

*/5 * * * * wget -q -O – https://itakeit.ro/wp-content/plugins/wisexml-connect/heartbeat.php >/dev/null 2>&1

I don't have any experience on this. Can somebody help me?

Thank you, Alin

PS: on the first cron job (I see that my post don't have the entire message): Minute Hour Day Month Weekday (there are * * * * *)

Thanks!

yagmoth555
  • 16,758
  • 4
  • 29
  • 50
Alin Horne
  • 1
  • 1
    If the job run more than one minute you will have a lot of jobs on the queue. Are you sure you want it every minute? Have you try to be every 10 minutes for example? – Romeo Ninov Dec 28 '22 at 08:29
  • Hi Romeo. Basically, I just heard about cron jobs. And I don't know how really works, just I have an ideea. I changed now to 5 minutes. Indeed, are now around 750 jobs in queue. And the cron job run only 10 max 15 jobs/hour. Even he finish them in 5-10 minutes, the rest of 50-55 minutes don't do anything :( – Alin Horne Dec 28 '22 at 11:05
  • The best you can do is to set the interval to 10 minutes to be sure they will finish (more or less) in this time. About old jobs in queue will be better to kill them. – Romeo Ninov Dec 28 '22 at 12:21
  • The PS is not required as the lines are formatted and the `*` are visible – Romeo Ninov Dec 28 '22 at 12:42
  • 1
    Thank you Romeo. I deleted all lines who are imported on 27.12. I have now around 300 jobs (from 750 total), imported today (28.12). I changed the time to */10 minutes. I need a couple of hours to check what happen. Few days ago I deleted many days pending in queue. One day have around 500 jobs/day. If in one hour are solved 10-15, this means for an average of 12/hour * 24 = 288 solved jobs, and each day are remaining 212 jobs unsolved. I hope by changing to 10 minutes, to solve the issue :) I will revert to you! – Alin Horne Dec 28 '22 at 13:08
  • The wise approach is to set the time between executions enough to finish the job. So if your job is between 7 and 12 minutes think about jobs each 15 minutes (just example) – Romeo Ninov Dec 28 '22 at 13:33
  • The current hour is passed already. The cron job solved 10 jobs from minutes 13:10:03 to 13:10:52 (so practically he run 1 minute only), then he started on 13:20:15 and run until 13:21:08 another 10 jobs (1 minute only). Is an improvement, because is first time when I see two runs in the same hour. Can I do something more, in order to run more than twice/hour? – Alin Horne Dec 28 '22 at 13:39
  • I have no idea what is this service and what is the usage of the results of this call. So I can't recommend actions. When I call it it return JSON file `{"Code":200,"Message":"Ok","Data":[]}` – Romeo Ninov Dec 28 '22 at 14:50
  • Hello Romeo, Please I need your support. The cron jobs are running on same rule. Once per hour, just 10 jobs/hour. Basically, I need please your help with this: What is the difference between these 2 commands: https://itakeit.ro/wcmipconnector/readfiles and /usr/bin/flock -n /tmp/wc_process_import_itakeit.ro.lockfile /usr/bin/curl https://itakeit.ro/wcmipconnector/readfiles >> /home/itakeit/public_html/wp-content/uploads/mip-connector/logs/process_import_cron.log Thank you very much, – Alin Horne Dec 29 '22 at 16:07
  • Alin, this is new question. Please create it here: https://superuser.com/questions/ask Meanwhile I will answer to this question. – Romeo Ninov Dec 29 '22 at 16:14

1 Answers1

0

Using flock make you "hold" cron tasks execution until release of lock file. And this is because the execution of one task take more than one minute. The best you can do is to create tasks to run every 10 minutes so you will never get them added to a queue.

Also check and measure the execution times and carefully set the interval between executions. Or take measures to make each execution independent.

Romeo Ninov
  • 5,263
  • 4
  • 20
  • 26