Python sys.argv[1:] not picking up command line options

Question

Update/Solution: the answer is below, from Zack. The problem was, indeed, DOS line endings on the script file itself, clenotes.cmd. Since I futzed with the various files so much, I deleted the whole directory and then re-downloaded a fresh copy from HERE. I ran Zack's perl script on the file just like so:

perl -pi.bak -e 's/[ \t\r]+$//' clenotes.cmd

I then edited the command execution just slightly so that the final script became:

CWD=`dirname $0`
JYTHON_HOME="$CWD"
LIB_DIR="$JYTHON_HOME/lib"
NOTES_HOME="/opt/ibm/lotus/notes/"
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$NOTES_HOME
java -cp "$LIB_DIR" -jar "$LIB_DIR/jython.jar" -Djython.home="$CWD/" -Dpython.path="$LIB_DIR:$CWD/ext" -Djava.library.path="$NOTES_HOME" "$LIB_DIR/clenotes/cletes/clenotes.py" "$@"

That was it -- everything else worked. No edits needed to clenotes.py or clenotes.cfg. Many thanks for sticking with the question, which I guess ended up being quite simple.

---

Update: I'm cutting down on some of the code to make this more readable and remove unnecessary information from the post.

---

I'm trying to get Lotus Notes command line to work on Linux and am having an issue with something related to sys.argv[1:] in the python file. The windows script is here:

@echo off
@setlocal 
set CWD=%~dp0
set JYTHON_HOME=%CWD%
set LIB_DIR=%JYTHON_HOME%/lib
java -cp %LIB_DIR% -jar %LIB_DIR%/jython.jar -Djython.home=%CWD%  -python.path=%LIB_DIR%;%CWD%/ext %LIB_DIR%/clenotes/clenotes.py  %*

@endlocal

I was having a tough time with variables, so for Linux, it simply looks like this:

java -cp ./lib/ -jar ./lib/jython.jar -Djython.home=./ -Dpython.path=./lib:./ext -Djava.library.path=/opt/ibm/lotus/notes/ ./lib/clenotes/clenotes.py $*

I run it from within the directory. In any case, what puzzles me is that it's not picking up any options I pass from the command line. clenotes.cmd --help results in

No commands specified. Use --help option for usage.

Here is the section where the command line arguments are supposed to be parsed:

def main():    

  Output.log("Entering %s v%s" % (PROGRAM_NAME,VERSION),Output.LOGTYPE_DEBUG)
  cliOptions2=[]
  for opt in cliOptions:
    opt2=opt.replace('--','')
    opt2=opt2.replace('!','=')
    cliOptions2.append(opt2)
  opts=[]
  args=[]
  try:
    opts, args = getopt.getopt(sys.argv[1:], '', cliOptions2)

I'm using Python 3.1.3 on Arch Linux 64bit in a 32bit chroot environment. Can I provide anything else?

Just in case it's needed... HERE is the whole clenotes.py file.

Also, as requested in the comments, the config file (which contains the help message and viable options/arguments, is HERE

---

Update

After a lot of fiddling, the best progress I have made has been to examine what it's setting as opts and args in the (main) method. Most surprising was that when passing an argument and then looking at it's parsed result using print sys.argv, the option would come up with a trailing \r in it. For example:

clenotes.cmd appointments
args is ['appointments\r']

On Windows I did the same and args was reported as ['appointments']. Furthermore, manually setting args=['appointments'] and then commenting out the section where getopt.getopt is assigning a value worked.

Lastly, I've found that when using multiple arguments, n-1 of them get interpreted and used while the nth one gets ignored. This is kind of a workaround since I can actually use the script... but obviously it's not preferred. If I want to look at today's appointments, I can execute clenotes.cmd appointments --today --today and it will work. sys.argv will spit out: ['appointments', '--today', '--today\r'].

So... what's causing the trailing \r? I'm thinking it has to do with the actual script. Note it again:

java -cp ./lib/ -jar ./lib/jython.jar -Djython.home=./ -Dpython.path=./lib:./ext -Djava.library.path=/opt/ibm/lotus/notes/ ./lib/clenotes/clenotes.py $*

So... bunch of path stuff and then the actual python file: clenotes.py $*

I got the $* from HERE

Is it picking up the carriage return??

Answer 1

I think your problem is that clenotes.cfg has DOS line endings, which Python is misinterpreting. Try changing this line of clenotes.py

config.readfp(open('%sconfig/clenotes.cfg' % System.getProperty('jython.home')))

to read

config.readfp(open('%sconfig/clenotes.cfg' % System.getProperty('jython.home'), "rU"))

The "rU" tells Python that even though it's running on a Unix system it should be prepared to cope with a file containing DOS line endings. See http://docs.python.org/library/functions.html#open -- scroll down to the paragraph that begins "In addition to the standard fopen() modes...".

(Or you could run this command: perl -pi.bak -e 's/[ \t\r]+$// clenotes.cfg -- that will convert it to Unix line endings. In your shoes I would probably do both.)

(If neither of the above suggestions helps, the next thing I would try is hitting clenotes.py itself with the above perl command. I don't see how that could be the problem, but if the \r characters are not coming from clenotes.cfg, the .py file is the only plausible remaining source.)

(EDIT: Based on your comments on the question itself, I now think it's clenotes.cmd, the shell script wrapper, that needs to be converted from DOS to Unix line endings.)

Answer 2

I'll have to keep looking to figure out where that \r is coming from. But in the meanwhile, this problem has become much simpler. Once the args are parsed, do this:

args = [arg.strip() for arg in args]

That will get rid of the \r

EDIT: But wait -- is this only a partial solution? Is it still not parsing options correctly?

EDIT2: Seems like the \r needs to be stripped earlier. When there's no command, the /r never gets stripped, because the above only strips \r after getopt is done. This should have been obvious to me before -- instead of passing sys.argv[1:] here

opts, args = getopt.getopt(sys.argv[1:], '', cliOptions2)

modify it first

argv = [arg.strip() for arg in sys.argv[1:]]
opts, args = getopt.getopt(argv, '', cliOptions2)

You could also just do sys.argv[-1] = sys.argv[-1].strip()... but the c programmer in me starts to feel a bit queasy looking at that. Probably irrational, I know.

Or just do what Zack said and convert clenotes.cmd to linux format -- however, note that stripping here will ensure that other people will not have to solve the same problem over again. (On the other hand, it's a little ugly, or at least mysterious to people not expecting such problems.)