1

I'm trying to run the curl command in a loop with a range of ports, such that the loop runs for 2000 times, and after every 5 source ports used the range is re-used from port number 4000. But after the first 5 ports, I'm seeing a bind failed error because curl keeps these old connections for a while. Is there any way I can remove old connections and again restart the loop?

for i in {1...2000}
do
curl --local-port 4000-4005 http://10.52.26.22 -m 1
sleep 1
done
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
Goldy Tomy
  • 85
  • 1
  • 9
  • It's tempting to call this an OS configuration problem rather than a problem with your script, and point you in the direction of [unix.se] to get guidance on which kernel settings to twiddle to moot the issue. – Charles Duffy Mar 06 '23 at 23:42
  • This is happening because local ports are reserved in `TIME_WAIT` state for a few minutes after the connection completes. – Barmar Mar 06 '23 at 23:43
  • (BTW, the curl documentation _does_ warn you about this effect). – Charles Duffy Mar 06 '23 at 23:44
  • This is a known issue with `curl`, see [this bug](https://github.com/curl/curl/issues/6288) – Barmar Mar 06 '23 at 23:44
  • Also see [What could cause so many TIME_WAIT connections to be open?](https://stackoverflow.com/questions/33177370/what-could-cause-so-many-time-wait-connections-to-be-open), which discusses the aforementioned kernel settings that can be tweaked as mitigation. – Charles Duffy Mar 06 '23 at 23:44
  • What's the use case for specifying the local ports like this? Is there a firewall that only allows connections from these ports? – Barmar Mar 06 '23 at 23:46
  • so that means cannot re-use the range of ports for the curl request? – Goldy Tomy Mar 06 '23 at 23:47
  • @GoldyTomy, eh, read the links above; "cannot" is an overstatement; you _can_ reconfigure your OS to make local ports more immediately reusable, but there are reasons that the default is what it is, so making those tweaks is sometimes unwise. – Charles Duffy Mar 06 '23 at 23:47
  • @charles Duffy, thanks I was able to solve this based on the above link and handling Time-wait. – Goldy Tomy Mar 06 '23 at 23:54
  • This might help: [Sending Curl Request with Close Connection Header](https://reqbin.com/req/c-ulfpjfcv/curl-command-with-close-connection-header) – Cyrus Mar 07 '23 at 00:05
  • @cyrus I tried using -H "Connection: close" but it doesn't increase the source port, it always re-uses the 1st port. So this isn't what I want, I want it to increase the port numbers as well. – Goldy Tomy Mar 07 '23 at 00:12
  • @GoldyTomy If you want to increase the port number, don't use a range, give a specific port. Put the port number in a variable that you increment and wrap around. – Barmar Mar 07 '23 at 00:31
  • (This smells to me like you're doing something like using policy routing to go out different gateways based on the port numbers? Cute). – Charles Duffy Mar 07 '23 at 18:51

1 Answers1

0

Given the question as clarified in comments, consider:

#!/usr/bin/env bash

local_port_min=4000
local_port_max=4005

for ((i=0; i<2000; i++)); do
  if (( local_port < local_port_min )) || (( local_port >= local_port_max )); then
    local_port=$(( local_port_min - 1 ))
  fi
  (( ++local_port ))
  curl -H "Connection: close" --local-port "$local_port" http://10.52.26.22 -m 1
  sleep 1
done
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441