Pass JSON to C# python process

Question

I have a little problem. I want to start a python process from C# and I need to pass some data to it. The data is in json format, but if I serialize the data from c# it look like this

"[{\"SearchTerm_id\":1,\"Term\":\"lorem ipsum\"},{\"SearchTerm_id\":2,\"Term\":\"lorem ipsum\"}]}"

and is not valid for python because of the scapation for double quote. How I can pass the data from c# to a python script ?

This is my code:

List<SearchTerms> searchTerms = await _context.SearchTerms.ToListAsync();
var json = JsonConvert.SerializeObject(searchTerms);

ProcessStartInfo processInfo = new ProcessStartInfo();
string scriptPath = Path.Combine(System.IO.Directory.GetCurrentDirectory(), "Python\\scrapeGoogle.py");
processInfo.FileName = "python3";
processInfo.Arguments = string.Format("{0} {1}", scriptPath, json);
processInfo.UseShellExecute = false;
processInfo.CreateNoWindow = true;
processInfo.RedirectStandardOutput = true;
processInfo.RedirectStandardError = true;
process.StartInfo = processInfo;
process.Start();

Possible duplicate of [Newtonsoft.Json SerializeObject without escape backslashes](https://stackoverflow.com/questions/20312974/newtonsoft-json-serializeobject-without-escape-backslashes) — pstrjds, Sep 30 '19 at 10:55

score 2 · Accepted Answer · answered Sep 30 '19 at 10:57

2

If you are viewing the result of var json = JsonConvert.SerializeObject(searchTerms); in the debugger. The debugger only shows the \ as a visual aid, representing the string just as it would have to be written in c#.

Try running Console.Write(json); and view the output.

The output won't contain the escape character. This is the true value.

answered Sep 30 '19 at 10:57

A Redfearn

506
5
15

You are right. So the problem seams to be elsewhere because if I have only a print(sys.argv[1]) in my python, I will recive my data as follow : "[{SearchTerm_id:1,Term:Lorem ipsun},{SearchTerm_id:2,Lorem Ipsum}]" – Filip Laurentiu Sep 30 '19 at 11:05
1

Yep must be somewhere in the python script, at least you can eliminate escape characters being the problem! – A Redfearn Sep 30 '19 at 11:16
@FilipLaurentiu Do you mind marking as the answer if it helped? – A Redfearn Sep 30 '19 at 14:55
1

Is not the final answer but it put me on the right way – Filip Laurentiu Sep 30 '19 at 15:25

gpph · Answer 2 · 2022-02-17T01:34:41.203

Just solved this for myself, so thought I would comment.

The Problem

@Filip Laurentiu (OP) correctly identified his problem when replying to @A Redfearn's answer.

You are right. So the problem seams to be elsewhere because if I have only a print(sys.argv1) in my python, I will recive my data as follow : "[{SearchTerm_id:1,Term:Lorem ipsun},{SearchTerm_id:2,Lorem Ipsum}]"

C# is actually formatting the JSON without escape characters, contrary to what the original question said. So it is doing what it is supposed to do. If you print the text, you see {"SearchTerm_id":1,"Term":"lorem ipsum"}.

However, Python is receiving the JSON without the double quotes. Printing from Python shows {SearchTerm_id:1,Term:lorem ipsum}. And when you call json.loads with that bad JSON, it throws.

json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

It seems that when you call Process.Start() in C#, the shell strips the double quotes from the JSON in your ProcessStartInfo.Arguments list. So Python receives the argument without double quotes.

The Solution

In C#, update the JSON string after serializing, so that you escape the double quotes. Here is some code to help with that.

using Newtonsoft.Json;
using System.Text;
public static class JsonHelper
{
    public static string ToJsonString(
        object obj,
        bool escapeDoubleQuotes = false)
    {
        string serialized = JsonConvert.SerializeObject(obj);
        if (escapeDoubleQuotes)
        {
            // e.g., '{"key":"value"}' -> '{\"key1\":\"value\"}'
            // Do this when need to pass json as cmd line arg via System.Diagnostics.Process. Else shell will strip the double quotes, so your new process might not know what to do with it
            serialized = serialized.Replace(@"""", @"\""");
        }

        return serialized;
    }
}

So to get his original code working, OP would just change

var json = JsonConvert.SerializeObject(searchTerms);

to

var json = JsonHelper.ToJsonString(searchTerms, escapeDoubleQuotes: true);

Since command line arguments have a number of limitations, I would pass the json through stdin. That way you should be able to stream everything without ever needing the entire json string in either C# or python. — Jeremy Lakeman, Feb 03 '22 at 03:07

score 0 · Answer 3 · answered Sep 30 '19 at 11:04

0

You can use messagePack library. I managed to pass strings and other types between python and c# with high performance.

answered Sep 30 '19 at 11:04

Lior

171
2
11

Pass JSON to C# python process

3 Answers3