Pass Binary string/file content from c++ to node js

Question

I'm trying to pass the content of a binary file from c++ to node using the node-gyp library. I have a process that creates a binary file using the .fit format and I need to pass the content of the file to js to process it. So, my first aproach was to extract the content of the file in a string and try to pass it to node like this.

   char c;
   std::string content="";
   while (file.get(c)){
    content+=c;
   }

I'm using the following code to pass it to Node

v8::Local<v8::ArrayBuffer> ab = v8::ArrayBuffer::New(args.GetIsolate(), (void*)content.data(), content.size());
args.GetReturnValue().Set(ab);

In node a get an arrayBuffer but when I print the content to a file it is different to the one that show a c++ cout.

How can I pass the binary data succesfully?

Thanks.

Is your binary data printable? A lot of *binary* data values are not printable. — Thomas Matthews, Aug 23 '17 at 20:53
Like @Thomas said, how does your _binary data_ look like? Do you use some encoding? — user0042, Aug 23 '17 at 20:55
You should *write* content of this buffer to the file instead of *printing* it. — user7860670, Aug 23 '17 at 20:56
I read the binary data from a binary file with this code: `char c; std::string content=""; while (file.get(c)){ content+=c; }` I need to pass the content of the file to node, is there any other way to do that? — jmahecha, Aug 23 '17 at 21:05
You should open file in binary mode and read entire file content using [`read`](http://en.cppreference.com/w/cpp/io/basic_istream/read) method. Though you can do this in node itself. — user7860670, Aug 23 '17 at 21:11
The library I'm using creates the binary file from a json file with some specifications, but I need to pass the content of the file to node instead of saving it from c++. The idea is to use the content of the file later in a larger aplication. — jmahecha, Aug 23 '17 at 21:17

paulsm4 · Answer 1 · 2017-08-23T21:48:47.507

1

Probably the best approach is to write your data to a binary disk file. Write to disk in C++; read from disk in NodeJS.
Very importantly, make sure you specify BINARY MODE.

For example:

myFile.open ("data2.bin", ios::out | ios::binary);
Do not use "strings" (at least not unless you want to uuencode). Use buffers. Here is a good example:

How to read binary files byte by byte in Node.js

var fs = require('fs');

fs.open('file.txt', 'r', function(status, fd) {
    if (status) {
        console.log(status.message);
        return;
    }
    var buffer = new Buffer(100);
    fs.read(fd, buffer, 0, 100, 0, function(err, num) {
        ...
    });
});

You might also find these links helpful:
- https://nodejs.org/api/buffer.html <= Has good examples for specific Node APIs
- http://blog.paracode.com/2013/04/24/parsing-binary-data-with-node-dot-js/ <= Good discussion of some of the issues you might face, including "endianness" and "interpreting numbers"

ADDENDUM:

The OP clarified that he's considering using C++ as a NodeJS Add-On (not a standalone C++ program.
Consequently, using buffers is definitely an option. Here is a good tutorial:

https://community.risingstack.com/using-buffers-node-js-c-plus-plus/

If you choose to go this route, I would DEFINITELY download the example code and play with it first, before implementing buffers in your own application.

edited Aug 23 '17 at 21:48

answered Aug 23 '17 at 21:28

paulsm4

114,292
17
138
190

Do you think that passing the buffer from the binary file is a bad idea? Do you think is better to save the file to disk in c++ and read it from node that try to pass the content? – jmahecha Aug 23 '17 at 21:33
Your Node program and your C++ programs are two different processes, running in two different address spaces, aren't they? If so, then "yes", a disk file is a good choice. Other [IPC](https://en.wikipedia.org/wiki/Inter-process_communication) choices besides "files" might include "sockets" and/or "pipes". – paulsm4 Aug 23 '17 at 21:36
The node-gyp library https://github.com/nodejs/node-gyp that build bundles for node js with othe code for example c++ – jmahecha Aug 23 '17 at 21:40
Fair enough: Writing a Node Add-on in C++ is also a viable alternative. Please read my post above, and read (and try the sample code) at this link: [Scott Frees: Using Buffers to share data between Node.js and C++](https://community.risingstack.com/using-buffers-node-js-c-plus-plus/) – paulsm4 Aug 23 '17 at 21:50
I check out you post and base on that I'm working on a example to pass a buffer using this code: `info.GetReturnValue().Set(Nan::NewBuffer(&buffer[0], size).ToLocalChecked());` I get data from node, but is not the same that in c++, maybe is because is binary data, I don't know is using this the data is transform o alter in some way. have you try ti use buffer with binary data? – jmahecha Aug 31 '17 at 19:29

score 0 · Answer 2 · answered Aug 23 '17 at 21:50

It depends but for example using redis

Values can be strings (including binary data) of every kind, for instance you can store a jpeg image inside a value. A value can't be bigger than 512 MB.

If the file is bigger than 512MB, then you can store it in chunks. But I wouldnt suggest since this is an in-memory data store

Its easy to implement in both c++ and node.js

Pass Binary string/file content from c++ to node js

2 Answers2