Inline python3 from golang using exec and the return result

Question

I've been following this SO solution to call python from golang and capture the return from go code to decode a simple csv and converting into json.

This works from the cli as the command:

$ python3.7 -c "exec(\"import csv,json  \na=list() \nfor i in csv.DictReader(open('/Users/Astra/data_in/Data.csv', 'r')): a.append(json.dumps(i))   \nb=dict(enumerate(a))   \nc=json.dumps(b, sort_keys=True, indent=4)   \nprint(c)\")"

From Golang the command set as follows:

cmd := exec.Command("python", "-c", "import csv,json; a=list(); for i in csv.DictReader(open('/Users/Astra/data_in/Data.csv', 'r')): a.append(json.dumps(i)); b=dict(enumerate(a)); c=json.dumps(b, sort_keys=True, indent=4); print(c)")

However, I'm getting the following error:

exit status 1
  File "<string>", line 1
    import csv,json; a=list(); for i in csv.DictReader(open('/Users/Astra/data_in/Data.csv', 'r')): a.append(json.dumps(i)); b=dict(enumerate(a)); c=json.dumps(b, sort_keys=True, indent=4); print(c)
                                                                                                                                                ^
SyntaxError: invalid syntax

I think I need to move back the indent parallel to the for loop but how's that possible from the stringified command?

thanks in advance.

UPDATE:

Below is the working python equivalent, the line assigning c variable needs to be along the for loop

a = list()
for i in csv.DictReader(open('/Users/Astra/data_in/Data.csv', 'r')):
    a.append(json.dumps(i))
c = json.dumps(dict(enumerate(a)), sort_keys=True, indent=4)
print(c)

May I ask why you're calling an external python process instead of parsing it directly in go? — Demian Brecht, Mar 12 '20 at 03:35
@DemianBrecht thanks for asking. It's an attempt at avoiding a huge struct definition for the csv I'm working on. I've no issue with struct as long as they are small and manageable but this 3 liner python does a great job of decoding. — systrigger, Mar 12 '20 at 04:03

tdelaney · Accepted Answer · 2020-03-12T04:03:05.240

If you run python -. It will read the program from stdin. This is useful for arbitrarily long python scripts that come from varied sources.

python_source = `import csv,json 
a=list()
for i in csv.DictReader(open('/Users/Astra/data_in/Data.csv', 'r')):
    a.append(json.dumps(i))
b=dict(enumerate(a))
c=json.dumps(b, sort_keys=True, indent=4)
print(c)`

 cmd := exec.Command("python", "-")
 cmd.Stdin = strings.NewReader(python_source)
 ...

score 0 · Answer 2 · answered Mar 12 '20 at 03:33

Try passing the argument with the linefeeds and correct indentation using a raw string literal:

cmd := exec.Command("python", "-c", `import csv,json; 
a=list(); 
for i in csv.DictReader(open('/Users/Astra/data_in/Data.csv', 'r')): 
   a.append(json.dumps(i)); 
   b=dict(enumerate(a)); 
   c=json.dumps(b, sort_keys=True, indent=4); 
   print(c)
`)

Inline python3 from golang using exec and the return result

2 Answers2