How to run curl commands parallel using GNU parallel

Question

Just recently started programming in bash. I want to run curl command for 'n' times where n is user input. And I want to run in parallel. I came across GNU Parallel. the curl code for running n times is

ip1="some ip"
ip2="some ip"
ip3="some ip"
for ((j=1; j<= $servers;))
do
    for ((i= 1; i<= $val; i++))
    do
        c=ip$j
        value=scatest"$j$i"
        curl -s -X POST ${!c}:4000/invoke -H "content-type: application/json" -d '{"args":["org1","'$value'","27-06-2018"]}'
        if ((i % $val == 0))
        then
            ((j++))
        fi
    done
done

i want to run this curl command equally and parallel on these different hosts based on the user input. Can any one help me out?

Thanks in advance

Ole Tange · Accepted Answer · 2018-07-03T22:19:17.470

It is fairly simple to parallelize per server:

val=1000
doit() {
    server="$1"
    serverno="$2"
    for ((i= 1; i<= $val; i++))
    do
        value=scatest"$serverno$i"
        curl -s -X POST ${server}:4000/invoke -H "content-type: application/json" -d '{"args":["org1","'$value'","27-06-2018"]}'
    done
}
export -f doit
export val
parallel -j0 doit {} {#} ::: serverip1 serverip2 ...

If you can accept scatest has a different value you can parallelize even further:

doit() {
    server="$1"
    jobno="$2"
    i="$3"
    value=scatest"$jobno$i"
    curl -s -X POST ${server}:4000/invoke -H "content-type: application/json" -d '{"args":["org1","'$value'","27-06-2018"]}'
}
export -f doit
seq $val | parallel -j0 doit {1} {#} {2} ::: serverip1 serverip2 ... :::: -

How to run curl commands parallel using GNU parallel

1 Answers1