Ruby UNIXServer with Node.js client and chunked data

Question

I've this Ruby server that uses a Unix Socket:

require 'socket'
require 'json'

server = UNIXServer.new('/tmp/ciccio.sock')
loop do
  sock = server.accept
  loop do
    begin
      data = sock.recv(1024)
      break if data.empty?
      # calculate a response
      response = {supermega: "very long json" * 10000}
      sock.write(response.to_json)
      sock.flush
    rescue Errno::EPIPE, Errno::ENOTCONN
      break
    end
  end
end

And I've this client in JavaScript that uses node.js net api:

Net = require('net');

var client = Net.connect({ path: '/tmp/ciccio.sock' }, () => {
    console.log('write data');
    client.write('hello world!');
});

client.on('data', (data) => {
    console.log(data.toString());
    var json = JSON.parse(data);
    // do something with json
});

client.on('end', () => {
    console.log('end');
});

client.on('error', (error) => {
    console.log(error.toString());
});

The problem is that if the data is big (over about 8193 characters) JSON.parse fail because data is chunked. How can I get the whole json as string and then parse it?

score 0 · Answer 1 · answered Dec 01 '15 at 13:04

0

You should only parse the JSON when you have received all of the text. Try this in the client.

Net = require('net');

var client = Net.connect({ path: '/tmp/ciccio.sock' }, () => {
    console.log('write data');
    client.write('hello world!');
});

// this will contain the whole JSON String
var totalString = '';

client.on('data', (data) => {
    // progressively add the chunks to totalString
    var chunk = data.toString();
    console.log(chunk);
    totalString += chunk;
});

client.on('end', () => {
    // now you may parse the JSON
    var json = JSON.parse(totalString);
    totalString = ''; // release totalString to the garbage collector
    // now do something with json
    console.log('end');
});

client.on('error', (error) => {
    console.log(error.toString());
});

Note this could be done much more efficiently if you had some way of predicting the size of the data. String appending with += is very inefficient.

answered Dec 01 '15 at 13:04

William Rosenbloom

2,506
1
14
37

The `end` event is never emitted in my example... I'm using net api (https://nodejs.org/docs/latest/api/net.html#net_event_end), not http api (https://nodejs.org/docs/latest/api/http.html#http_response_end_data_encoding_callback) – Pioz Dec 01 '15 at 13:11
@Pioz Now looking at your Ruby server, you never call `sock.close`. Have you tried doing that? – William Rosenbloom Dec 01 '15 at 13:14
I don't want close my socket, I want to keep it opened... I don't want make a connection for every message. – Pioz Dec 01 '15 at 13:16
1

@Pioz Ok so the thing you are calling `sock` is not a socket. It's a temporary connection on that socket. There will be no problem with closing it. Your AF_UNIX socket is held in the variable you have named `server`. – William Rosenbloom Dec 01 '15 at 13:18

Ruby UNIXServer with Node.js client and chunked data

1 Answers1