I want to convert this C++ project (Facebook FastText)
├── args.cc
├── args.h
├── dictionary.cc
├── dictionary.h
├── fasttext.cc
├── fasttext.h
├── main.cc
├── matrix.cc
├── matrix.h
├── model.cc
├── model.h
├── productquantizer.cc
├── productquantizer.h
├── qmatrix.cc
├── qmatrix.h
├── real.h
├── utils.cc
├── utils.h
├── vector.cc
└── vector.h
I want to run in node.js
. I have first built it as a project with emmake make
and I have got the linked LLVM object files:
-rw-r--r-- 1 loretoparisi staff 27536 30 Lug 10:20 args.o
-rw-r--r-- 1 loretoparisi staff 78632 30 Lug 10:20 dictionary.o
-rw-r--r-- 1 loretoparisi staff 23864 30 Lug 10:20 productquantizer.o
-rw-r--r-- 1 loretoparisi staff 12120 30 Lug 10:20 matrix.o
-rw-r--r-- 1 loretoparisi staff 9132 30 Lug 10:20 qmatrix.o
-rw-r--r-- 1 loretoparisi staff 10532 30 Lug 10:20 vector.o
-rw-r--r-- 1 loretoparisi staff 30036 30 Lug 10:20 model.o
-rw-r--r-- 1 loretoparisi staff 1616 30 Lug 10:20 utils.o
-rw-r--r-- 1 loretoparisi staff 118404 30 Lug 10:20 fasttext.o
-rwxr-xr-x 1 loretoparisi staff 270940 30 Lug 10:20 fasttext
but clearly this was not bit code! I have then compiled with emmake make VERBOSE=1
and it turns out it was not using em++
compiler, so I have tried cmake
:
emconfigure cmake
emmake make VERBOSE=1
At this point I can see em++
running
/usr/local/Cellar/cmake/3.11.1/bin/cmake -E cmake_link_script CMakeFiles/fasttext-shared.dir/link.txt --verbose=1
/Users/loretoparisi/Documents/MyProjects/emsdk/emscripten/1.38.10/em++ -fPIC -pthread -std=c++11 -funroll-loops -O3 -march=native -shared -Wl,-soname,libfasttext.so -o libfasttext.so @CMakeFiles/fasttext-shared.dir/objects1.rsp
so I get then the following output:
-rw-r--r-- 1 loretoparisi staff 870544 30 Lug 10:40 libfasttext.a
-rw-r--r-- 1 loretoparisi staff 306646 30 Lug 10:40 fasttext.wasm
-rw-r--r-- 1 loretoparisi wheel 114099 30 Lug 10:40 fasttext.js
-rw-r--r-- 1 loretoparisi staff 870504 30 Lug 10:40 libfasttext_pic.a
-rw-r--r-- 1 loretoparisi staff 663728 30 Lug 10:40 libfasttext.so
This turns out to be a valid javascript file, but it causes an exception when running it
$ node fasttext.js predict-prob /root/lang_id.bin - 2
exception thrown: 5280232 - Exception catching is disabled, this exception cannot be caught. Compile with -s DISABLE_EXCEPTION_CATCHING=0 or DISABLE_EXCEPTION_CATCHING=2 to catch.
while it should get from stdin
like
./fasttext predict-prob /root/lang_id.bin - 2
ja das is seher gut heute
__label__deu 0.999483 __label__bar 4.1711e-05
May this error due to stdin
in node.js? The command like seems to be fine, since it accepts and parses the parameters in the right way:
$ node fasttext.js predict-prob
usage: fasttext predict[-prob] <model> <test-data> [<k>] [<th>]
<model> model filename
<test-data> test data filename (if -, read from stdin)
<k> (optional; 1 by default) predict top k labels
<th> (optional; 0.0 by default) probability threshold
I have then modularized it like emcc -s MODULARIZE=1 -s LEGACY_VM_SUPPORT=1 -s WASM=0 -O1 libfasttext.so -o libfasttext.js
, and called within a node module like:
let FastTextModule = require('./fasttext-module.js');
let Module = null;
if (Module) {
return;
}
Module = {
noExitRuntime: true,
noInitialRun: false,
preInit: [],
preRun: [],
postRun: [function () {
console.log(`Loaded Javascript Module OK`);
}],
locateFile: "./",
arguments: ['predict-prob', '/root/lang_id.bin', '-', 2]
};
Module['locateFile'] = function (path, prefix) {
return prefix + path;
}
FastTextModule(Module);
But I'm getting the same error: exception thrown: 5278216 - Exception catching is disabled, this exception cannot be caught. Compile with -s DISABLE_EXCEPTION_CATCHING=0 or DISABLE_EXCEPTION_CATCHING=2 to catch.
In the same way I can run it setting noInitialRun:true
and then doing
Module.callMain(['predict-prob', '/root/ft_lang_model.bin', 'test.csv', '2']);
but having that error anyways.