applescript pass text to python script---bits of stdout get truncated

Question

I'm trying to use applescript to pass strings to a python script (my ultimate use is to process notes from icloud). For some reason, however, when I try to test things with print statements, it produces bizarre results.

Here's the applescript:

set s to "here is a

long string

with

line breaks"

do shell script "python t3.py " & quoted form of s

and here's t3.py:

import sys 
print("about to print whole argument list") 
print(sys.argv)
print("printed whole argument list")

when I call the applescript that calls the python script, this prints something really bizarre:

printed whole argument listng string\n\nwith\n\nline breaks']

If I comment out the last line of the python script, however, it prints: ['t3.py', 'here is a \n\nlong string\n\nwith\n\nline breaks'] which is closet to correct (it merely drops the first of the expected prints).

My first hypothesis was that this is some kind of stream buffering thing from the Python side, so I added flush=True to each of the print calls. No change in output.

What on earth is going on here? I'm using python 3.6.4.

I suspect that newlines in the output from the script are getting converted to carriage returns, probably by AppleScript. Exactly how are you capturing and viewing the output? — Gordon Davisson, Aug 05 '18 at 05:20
I'm simply running the applescript in a terminal window (via osascript, in iterm) and watching the output. — Paul Gowder, Aug 05 '18 at 15:10

Gordon Davisson · Accepted Answer · 2018-08-08T01:34:51.850

You're running into trouble with inconsistent encoding of line breaks in text. Different OSes indicate the ends of lines in text differently: unix (and unix derivatives like macOS) use the newline character (sometimes written as \n); DOS (and derivatives line Windows) use newline followed by carriage return (\n\r); and old-school Mac OS (before OS X) used just carriage return (\r).

AppleScript dates from the pre-OS X days of Mac OS, and still uses carriage returns. And it sometimes translates to/from the unix convention when talking to the rest of the OS, but not always. What's happening here is that your python script is producing output with newlines, AppleScript's do shell script command is capturing its output and converting to the carriage return convention, and it's never getting converted back. When that gets sent to the Terminal, the carriage returns make it go back to column 1, but not to the next line, so each "line" of output gets printed over top of the last one.

How to fix this (or whether it even needs fixing) depends on the larger context, i.e. what you're actually going to do with the output. In many contexts (including just running it on the command line), you can pipe the output through tr '\r' '\n\ to translate carriage returns in the output back into newlines:

$ osascript t3.applescript 
printed whole argument listg string\n\nwith\n\nline breaks']
$ osascript t3.applescript | tr '\r' '\n'
about to print whole argument list
['t3.py', 'here is a\n\nlong string\n\nwith\n\nline breaks']
printed whole argument list

EDIT: as for how to get AppleScript to produce results with unix-style delimiters... I don't see a simple way, but you can use the text replacement function from here to convert from CR to LF:

on replaceText(find, replace, subject)
    set prevTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to find
    set subject to text items of subject

    set text item delimiters of AppleScript to replace
    set subject to subject as text
    set text item delimiters of AppleScript to prevTIDs

    return subject
end replaceText


set s to "here is a

long string

with

line breaks"

set CRstring to do shell script "python t3.py " & quoted form of s

set LFstring to replaceText("\r", "\n", CRstring)

You could also make a special-purpose function:

on CR2LF(subject)
    set prevTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to "\r"
    set subject to text items of subject

    set text item delimiters of AppleScript to "\n"
    set subject to subject as text
    set text item delimiters of AppleScript to prevTIDs

    return subject
end CR2LF

Man. I did not know applescript was still in carriage return land. Thanks. Do you know how to force applescript to convert captured/passed-on stdout to proper line breaks? Other than, I suppose, just wrapping it up in a bash script or something to always pipe it through to tr like in your example. :-) — Paul Gowder, Aug 06 '18 at 14:25

applescript pass text to python script---bits of stdout get truncated

1 Answers1