Interprocess communication Unity3d python

Question

I am trying to receive data from my python script using Processes (IPC). Is it some sort of method for reading dictionaries in Unity from python ??? i am using Processes because speed is critical in my situation. Ideally i would like to send an image from unity and get results based on that from python. Thank you in advance

Unity code:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Text;

public class Main : MonoBehaviour
{
   // Start is called before the first frame update
  void Start()
{
    Console.WriteLine("Execute python process...");
    Option1_ExecProcess();
}

// Update is called once per frame
void Update()
{
    
}

static void Option1_ExecProcess()
{
    // 1) Create Process Info
    var psi = new ProcessStartInfo();
    psi.FileName = @"python.exe";

    // 2) Provide script and arguments
    var script = @"pyScript.py";


    psi.Arguments = $"\"{script}\"";

    // 3) Process configuration
    psi.UseShellExecute = false;
    psi.CreateNoWindow = true;
    psi.RedirectStandardOutput = true;
    psi.RedirectStandardError = true;

    // 4) Execute process and get output
    var errors = "";
    var results = "";

    using (var process = Process.Start(psi))
    {
        errors = process.StandardError.ReadToEnd();
        results = process.StandardOutput.ReadToEnd();
    }
}

}

Python script:

import logging
import math
import time
import traceback
import numpy as np
import io
from numpy import asarray, expand_dims
import tensorflow as tf
from tensorflow import keras
import base64
from tensorflow.python.keras.utils  import data_utils
import os

def closestDivisors(num):
n = math.floor(math.sqrt(num))
for i in range(n, 0, -1):
    if (num) % i == 0:
        res = [i, (num)//i]
        break
return res

def fixed_length_str(i):
return '%02d' % i

layers_indices = []
data = {}

def init():
image_model = tf.keras.applications.VGG16(include_top=True,
                                          weights='imagenet', pooling='avg')
for index, layer in enumerate(image_model.layers):
    if 'fc' in layer.name:
        layers_indices.append(index)
    if 'dense'  in layer.name:
        layers_indices.append(index)
    if 'conv' in layer.name:
        layers_indices.append(index)
    if 'predictions' in layer.name:
        layers_indices.append(index)
outputs = [image_model.layers[i].output for i in layers_indices]
image_model = tf.keras.Model(inputs=image_model.inputs, outputs=outputs)
return image_model, layers_indices

def return_data(image_model,layers_indices):
   try:
        img = tf.keras.preprocessing.image.load_img('p4.png', target_size=(224, 224))
        img = tf.keras.preprocessing.image.img_to_array(img)  # convert to tensor
        img = expand_dims(img, axis=0)

        feature_maps = image_model.predict(img)
        layer_counter = -1
        for fmap in feature_maps:  # of each layer
            feature_map = {}
            layer_counter = layer_counter + 1
            ix = 1
            layer_name = image_model.layers[layers_indices[layer_counter]].name
            if 'conv' in layer_name:
                x, z = closestDivisors(
                    image_model.layers[layers_indices[layer_counter]].output_shape[3])

                for i in range(x):
                    for j in range(z):
                        imageArray = fmap[0, :, :, ix-1]  # images  ndarray ix-1
                        imageArray = imageArray - imageArray.min()
                        if imageArray.max() != 0:
                            imageArray = imageArray/imageArray.max() * 255.0
                        imageArray = imageArray.astype(np.uint8)
                        var_name_conv = 'fmap_{}_{}'.format(
                            layer_name, fixed_length_str(i) +""+ fixed_length_str(j))
                        feature_map[var_name_conv] = imageArray.tolist()
                        ix += 1
            elif 'fc' in layer_name:
                x, z = closestDivisors(
                    image_model.layers[layers_indices[layer_counter]].output_shape[1])
                fmap = fmap - fmap.min()
                if fmap.max() != 0:
                    fmap = fmap/fmap.max() * 255.0
                fmap = fmap.astype(np.uint8)
                var_name_fc = 'fmap_{}'.format(
                    layer_name)
                feature_map[var_name_fc] = fmap.tolist()
            elif 'predictions' in layer_name:
                x, z = closestDivisors(
                    image_model.layers[layers_indices[layer_counter]].output_shape[1])
                fmap = fmap - fmap.min()
                if fmap.max() != 0:
                    fmap = fmap/fmap.max() * 255.0
                fmap = fmap.astype(np.uint8)
                var_name_pred = 'fmap_{}'.format(layer_name) #, fixed_length_str(x), fixed_length_str(z)
                feature_map[var_name_pred] = fmap.tolist()
            data[str(image_model.layers[layers_indices[layer_counter]].name)] = feature_map
        return data
   except:
        pass


def __init__(self):
  model, layers = init()
  return return_data(model, layers)

You could encode them as JSON or as a struct – mousetail Jul 19 '20 at 17:30 — mousetail, Jul 19 '20 at 17:30

score -1 · Answer 1 · answered Jul 19 '20 at 17:39

There are many ways to send data over a input stream. Standard out, just like the internet is just a way to send bytes from one process to another. There are many ways to send something like a dictionary over a data stream.

One is JSON, very common on the internet. It is a text-based format. This means that it takes more data on the stream to send information, and also encoding and decoding will be slower. However, it's much easier to work with. Python includes the json module for easily converting to and from json.

If you find that the speed that can be offered by json is too slow, you could design a costum-byte packed data format. This format can be much more compact since you don't need to include the names of keys and can use fixed lengths for numbers instead of the decimal representations. It will be much faster to pack and to unpack too since the data is already in a native data format. The struct module in python allows you to convert basic primitives (integers, strings) into struct representations, however you will need to combine them to encode the full dictionary yourself.

Which option is best is up to you. There are many many other ways to do this, look up data transfer protocols if you want more inspiration.

Interprocess communication Unity3d python

1 Answers1