Capturing telnet session - pexpect and telnetlib too slow

Question

I want to pull connection tables from a firewall. In some cases it can be more than 200k lines of

"TCP outside 46.33.77.20:53415 inside 10.16.25.63:80, idle 0:00:04, bytes 3230, flags UIOB"

and the like.

I've tried to implement both pexpect and telnetlib in order to grab these tables. Unfortunately both timeout and/or die with anything greater than 40k.

pexpect implementation:

connect.send("sho conn\n")
connect.expect("<--- More --->", timeout=360)
tmp_txt = connect.before

telnetlib implementation:

telnet.write("sho conn\n")
tmp_text = telnet.read_until("<--- More --->")

Is there a more robust method of grabbing this information? I control the number of lines given at a time with a pager value (prior to running this). Also - I'm monitoring the cpu on the firewall, so I know it's displaying the connections. Either there are too many or it's too fast for pexpect or telnetlib to keep up.

Thanks.

It sounds like this firewall is an appliance, not a standard PC running Linux. What kind of machine is this? — tMC, Apr 03 '12 at 19:22
Are you 100% sure the firewall is actually delivering all the data? Maybe its buffer is overflowing and you are never getting the More prompt. I'd try setting the pager limit down until it does work then teaching your script to press Space to see the next page. — Nick Craig-Wood, Apr 03 '12 at 19:23
@NickCraig-Wood - Yes, if I manually login to the device, set the pager to 0 and show the table, it spits it all out. I've thought about doing a space approach - but for whatever reason, the firewall takes a larger CPU hit when done like that. — BDub, Apr 03 '12 at 19:29
@tMC - It's a Cisco ASA (55xx series). I was previously looking to obtain the connection table via snmp - but could find a way to grab it in that way. — BDub, Apr 03 '12 at 19:35
in the examples you have above, you don't have the command to set the terminal size. have you tried sending `terminal length 0` (or ASA's equivalent) than just reading until you see the cmd prompt? — tMC, Apr 03 '12 at 19:39
@tMC - I didn't include that code, but yes, I am manipulating the terminal pager value. That's how I know it works right up until about 40k connections. — BDub, Apr 03 '12 at 19:42
I think what i would do is setup wireshark to watch the traffic (telnet is clear text) and see if the data is leaving the device and just being ignored by the telnetlib; or if its never getting transmitted. That said; I think snmp would return data in a for more formatted manner. Have you looked at a python-snmp lib? http://pysnmp.sourceforge.net/ Else, if you're running linux, you could call the `snmpwalk` command via `subprocess.Popen`. — tMC, Apr 03 '12 at 19:48

score 0 · Answer 1 · answered Sep 20 '12 at 13:18

It looks like your approach is fine to me. I would also page the output (to keep firewall CPU low) and then capture the output a screen full at a time.

If you are running into timeout errors then why not modify your expect to be a loop that expects each line or specific lines of output (I presume it has a regular format) and then only send space when it gets the "more" line for the next screen. I've used this pattern a lot to deal with long streams of output that may pause at different places.

You mention that the python process dies, we can't help you there - unless you are more detailed about what exception is being raised.

Capturing telnet session - pexpect and telnetlib too slow

1 Answers1