1

I'm trying to automate the download of Argos data using Python's telnetlib, but I can't seem to figure out how to get it to download all of the output. Part of my problem may be that I don't really understand the seemingly asynchronous nature of the commands.

Here's the code:

tn = telnetlib.Telnet(host = HOST, timeout = 60)
with open("argos_prv_{0}-1.txt".format(now_str), 'w') as of:
    tn.read_until("Username: ")
    tn.write(user + "\n")
    tn.read_until("Password: ")
    tn.write(password + "\n")
    tn.read_until("/")
    # Here's the command I'm trying to get the results of:
    tn.write("prv,,ds,{0:d},009919,009920\n".format(start_doy))
    # At this point, it's presumably dumped it all
    tn.read_until("ARGOS READY")
    tn.read_until("/")
    # Logging out
    tn.write("lo\n")
    lines = tn.read_all()
    of.write(lines)
    of.flush()

The code seems to run just fine, but when I look at the output file, it never has everything in it, cutting out at some random point. When I type the same commands in a real telnet session, it works just fine.

I get the sense it has something to do with trying to read_all() after logging out (tn.write("lo\n")), but when I look at the example documentation for telnetlib, it pretty much looks just like this.

Anyway, my question is: can anyone see what I'm doing wrong here? I want to grab the results of the prv,,ds command, but I'm only getting some of it using this particular code.

Thanks.

cswingle
  • 585
  • 1
  • 8
  • 13

1 Answers1

0
# At this point, it's presumably dumped it all
tn.read_until("ARGOS READY")
tn.read_until("/")

At a guess, this bit is sucking up the data and doing nothing with it. Think of it like a pair of pipes - you send stuff one way with write, and pull stuff back with read_*. If you've already sucked the stuff up, it won't still be waiting in the pipe when you do read_all later.

EDIT: OK, you're seeing a different problem. Try this:

lines = tn.read_until("ARGOS READY")
lines += tn.read_until("/")
tn.write("lo\n")
# Write out lines to file.
Thomas K
  • 39,200
  • 7
  • 84
  • 86
  • That would make sense if this were synchronous, but if you look at the telnetlib example (http://docs.python.org/library/telnetlib.html#telnet-example) they're doing the same thing (reading the result of the `ls` command after sending `exit`). – cswingle Jun 07 '11 at 16:43
  • It's not sending exit that's the problem. The key is that that example doesn't call `read_*` before it wants to get the data. So the data is still waiting when it calls `read_all` later. Try wrapping those two lines in `of.write(...)`. – Thomas K Jun 07 '11 at 16:52
  • I see what you mean. I tried commenting out `tn.read_until("ARGOS READY")` and `tn.read_until("/")` (so the code writes `prv,,ds` then `lo`, then `read_all()`), but it does the same thing: I get some of the `prv` output, but the connection seems to terminate before I get it all. – cswingle Jun 07 '11 at 17:14
  • There's something about not waiting long enough between the data request command `prv` and the logout command `lo`. If I comment out the `lo` command, the telnetlib debug output shows that I'm receiving all of the output, but since I'm not logging out, the script blocks at `read_all` until the connection is terminated, at which point I don't have the data at all. It's like I need to send `prv`, wait 59 seconds, send `lo`, then `read_all()` (except this doesn't work either -- I tried it) – cswingle Jun 07 '11 at 17:26
  • Thanks. I had already tried this (and pretty much all combinations including `read_until()` and `read_all()` with writing lines at various points). When I do just what you suggest, all I get are the `ARGOS READY` and `/` lines in the output. I get the feeling that `read_until` doesn't actually read everything in between, but only returns the line you're looking for. The rest is sitting in a buffer, but I think that buffer isn't big enough for me to grab everything with `real_all()`, or it gets truncated when the connection terminates as a result of the `lo` command. – cswingle Jun 08 '11 at 15:09
  • @cswingle: OK, I'm out of ideas then. I don't have a telnet server handy to test on. – Thomas K Jun 08 '11 at 17:18
  • @Thomas K: Thanks for the suggestions. I may have to set up my own telnet server just to figure it out. – cswingle Jun 08 '11 at 21:25