Why does my Node.js gRPC client take 3 seconds to send a request to my Python gRPC server?

Question

Let's begin by acknowledging that I am a gRPC noob. If I've asked a stupid question, go ahead and let me know, but only after explaining why it's stupid. Thanks! :)

Overview

I am developing an application that processes images. The variables that affect the result of the processing are to be changeable by the user. I wanted to provide a good-looking GUI while maintaining cross-platform compatibility. Thus, I decided to implement my image processing in Python, and use Node.js and Electron for my GUI.

Needing a way to send and receive data to and from my Python backend, I decided to use gRPC. Thus I have a Node.js gRPC Client paired with a Python gRPC Server.

After reading through many tutorials and learning a bit about protocol buffers, I was able to successfully transfer data between the two programs.

The Setup

The Node.js Electron app takes input from the user, and sends a request to the Python backend. The size of the request is a small object:

// From My Protocol Buffer
message DetectionSettings {
    float lowerThreshold = 1;
    float upperThreshold = 2;
    float smallestObject = 3;
    float largestObject  = 4;
    float blurAmount     = 5;

    int64 frameNumber    = 6;
    string streamSource  = 7;
}

Upon receiving this request, the Python application reads a frame from the specified streamSource and processes it. This processed frame is then converted to JPEG format and returned via gRPC to the Node.js app as an Image:

// From My Protocol Buffer
message Image {
    bytes image_data = 1;
    int32 height = 2;
    int32 width = 3;
    int64 frame = 4;    
}

The Problem

I've noticed that there is a variable latency between the time the request is made and the image is received. This time varies from a few ms up to nearly 3 SECONDS! After profiling the Python code, I've determined that the processing time is negligible. In addition, the time that it takes to return the Image via gRPC is also negligible.

Thus there is an intriguing delay between the RPC execution and the Python application receiving the call. Here are some logs with the times to better explain what's happening (The number in brackets is in seconds):

/* Node.js */
[160408.072] "Sending Request..."
[160408.072] "Executing RPC"
[160408.072] "RPC Executed"

/* Python */
[160411.032] [ py-backend ] Getting frame

/* Node.js */
[160411.617] "Got Data"

You can see that in this case the time from Node.js RPC execution to the Python method being called was around 3 seconds, while the time from the execution of the Python method to the Node.js application receiving the Image is less than 1 second 0.o

The Code

Python gRPC Server

# Create the server
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))

# Add our service to the server
processor_pb2_grpc.add_ProcessorServicer_to_server(
    ProcessorServicer(), server
)

server.add_insecure_port('[::1]:50053')
server.start()
server.wait_for_termination()

Node.js Client:

const grpc = require('grpc')
const PROTO_PATH = '../processor.proto'

const serviceConfig = {
    "loadBalancingConfig": [ {"round_robin": {} } ]
}
const options = {
    'grpc.service_config': JSON.stringify(serviceConfig)
}

const ProcessorService = grpc.load(PROTO_PATH).Processor
const client = new ProcessorService ('[::1]:50053',
    grpc.credentials.createInsecure(), options);

module.exports = client

Protocol Buffer:

syntax = "proto3";

message Number {
    float value = 1;
}

message Image {
    bytes image_data = 1;
    int32 height = 2;
    int32 width = 3;
    int64 frame = 4;    
}

message DetectionSettings {
    float lowerThreshold = 1;
    float upperThreshold = 2;
    float smallestObject = 3;
    float largestObject  = 4;
    float blurAmount     = 5;

    int64 frameNumber    = 6;
    string streamSource  = 7;
}

service Processor{
    rpc GetFrame(DetectionSettings) returns (Image) {};
}

Node.js gRPC Call:

function GetFrameBytes(detectionSettings, streamSource, frameNumber, callback)
{
    detectionSettings["streamSource"] = streamSource;
    detectionSettings["frameNumber"] = frameNumber;

    client.GetFrame(detectionSettings, (error, response) =>
    {
        if(!error){
            callback(response)
        }
        else
        {
            console.error(error);
        }
    });
}

tl;dr

Why on earth does it take so long for my Node.js client request to trigger my Python code?

Answer 1

There are a couple of factors that could be impacting the long time you are seeing.

First, the first request made by a client always takes longer because it has to do some connection setup work. The general expectation is that a single client will make many requests, and that the setup time will be amortized over all of those requests.

Second, you mentioned that you are using Electron. There have been some reports of gRPC for Node performing poorly when used in the Electron rendering process. You may see different results when running your code in the main process, or in a regular Node process.

You may also have different luck if you try the package @grpc/grpc-js, which is a complete reimplementation of the Node gRPC library. The API you are currently using for loading the .proto packages does not exist in that library, but the alternative using @grpc/proto-loader works with both implementations, and the other APIs are the same.