Adding a random delay for a Linux command

Question

I am using func to perform parallel commands on our servers.

The other day, we had an issue when a service restart of puppet via func made all our severs hit our puppetmaster at the same time.

My question: How can I execute the same exact command on a set of servers while adding a delay before it's executed on the individual servers?

E.g.: random_delay && service puppet restart

I am interested in the random_delay part of the command.

score 44 · Accepted Answer · edited Jan 07 '12 at 03:11

44

sleep $((RANDOM % MAXWAIT)) where MAXWAIT is the maximum desired delay in seconds.

edited Jan 07 '12 at 03:11

Jeff Ferland

20,547
2
62
85

answered Jan 07 '12 at 02:42

S19N

1,803
1
19
28

1

Seems to get stuck on "1" a lot if you echo that out... Thoughts? – Corey S. Jan 07 '12 at 03:00
1

Noticed this as well. Try this, for x in 1 2 3 4 5; do RANDOM=$x; i=$((RANDOM%10)); echo $i; sleep $i; done Obviously, MAXWAIT in S19N's answer is similar in effect to 10 in my answer. Change the RANDOM seed and add puppet code to whatever you desire of course. – dtbnguyen Jan 07 '12 at 03:11
3

@CoreyS. $RANDOM is seeding off the last value, hence the looping (I've gotten stuck on 5/6/7, 2/7, and 5). `sleep $((RANDOM % MAXWAIT))` is the correct way. I'll edit the answer as such. – Jeff Ferland Jan 07 '12 at 03:11
1

This is called *splay* in Pro Puppet. It is used to prevent a *thundering herd*, where many machines are doing the same thing simultaneously. In addition to this, I had success with cron scheduling using `fqdn_rand`. – François Beausoleil Jan 17 '12 at 13:46
2

This seems to depend on bash-specific extensions. In `dash` `echo sleep $((RANDOM % 900))` returns `sleep 0`. This means it is not safe to use in `#!/bin/sh` scripts or places like crontab. (The problems seems to be that `dash` does not understand `$RANDOM`) – Gert van den Berg Nov 22 '18 at 12:28

score 5 · Answer 2 · edited Sep 01 '21 at 09:42

5

sleep $((RANDOM))

The RANDOM will return a value between 0 and 32767 If you need to set a lower and upper limit to your sleep you need to define two other variables as follows:

MINWAIT=10
MAXWAIT=30
sleep $((MINWAIT+RANDOM % (MAXWAIT-MINWAIT)))

edited Sep 01 '21 at 09:42

Gerald Schneider

23,274
8
57
89

answered Sep 01 '21 at 07:56

KJA

51
1
2

1

With `MINWAIT=10` and `MAXWAIT=11` (only 1 unit appart), the command `echo $((MINWAIT+RANDOM % (MAXWAIT-MINWAIT)))` will always display `10` (id. `$MINWAIT`). – CDuv Oct 05 '21 at 19:22

score 2 · Answer 3 · answered Mar 05 '20 at 02:04

2

0 */12 * * * root perl -e 'sleep int(rand(43200))' && service puppet restart

This will sleep a random amount between 0 seconds and 12 hours (43200 = 12 x 60 x 60) every 12 hours

answered Mar 05 '20 at 02:04

rubo77

2,469
4
34
66

score 2 · Answer 4 · answered Jan 07 '12 at 11:41

I quite like S19N's innovative, yet less than ideal solution. I only say it's less than ideal, as it's still largely non-deterministic when stuff will actually happen. I'd much rather be able to guarantee when stuff will happen, and what stuff will happen when it does.

Puppet orchestration is actually a hard problem.
One of the "best-practice" solutions is to use MCollective which will not only allow you to configure when puppet runs on your cluster of machines, but you can also use it for other similar orchestration problems.

Adding a random delay for a Linux command

4 Answers4

Linked

Related