3

I want to convert this C++ project (Facebook FastText)

├── args.cc
├── args.h
├── dictionary.cc
├── dictionary.h
├── fasttext.cc
├── fasttext.h
├── main.cc
├── matrix.cc
├── matrix.h
├── model.cc
├── model.h
├── productquantizer.cc
├── productquantizer.h
├── qmatrix.cc
├── qmatrix.h
├── real.h
├── utils.cc
├── utils.h
├── vector.cc
└── vector.h

I want to run in node.js. I have first built it as a project with emmake make and I have got the linked LLVM object files:

-rw-r--r--   1 loretoparisi  staff   27536 30 Lug 10:20 args.o
-rw-r--r--   1 loretoparisi  staff   78632 30 Lug 10:20 dictionary.o
-rw-r--r--   1 loretoparisi  staff   23864 30 Lug 10:20 productquantizer.o
-rw-r--r--   1 loretoparisi  staff   12120 30 Lug 10:20 matrix.o
-rw-r--r--   1 loretoparisi  staff    9132 30 Lug 10:20 qmatrix.o
-rw-r--r--   1 loretoparisi  staff   10532 30 Lug 10:20 vector.o
-rw-r--r--   1 loretoparisi  staff   30036 30 Lug 10:20 model.o
-rw-r--r--   1 loretoparisi  staff    1616 30 Lug 10:20 utils.o
-rw-r--r--   1 loretoparisi  staff  118404 30 Lug 10:20 fasttext.o
-rwxr-xr-x   1 loretoparisi  staff  270940 30 Lug 10:20 fasttext

but clearly this was not bit code! I have then compiled with emmake make VERBOSE=1 and it turns out it was not using em++ compiler, so I have tried cmake:

emconfigure cmake
emmake make VERBOSE=1

At this point I can see em++ running

/usr/local/Cellar/cmake/3.11.1/bin/cmake -E cmake_link_script CMakeFiles/fasttext-shared.dir/link.txt --verbose=1
/Users/loretoparisi/Documents/MyProjects/emsdk/emscripten/1.38.10/em++ -fPIC  -pthread -std=c++11 -funroll-loops -O3 -march=native  -shared -Wl,-soname,libfasttext.so -o libfasttext.so @CMakeFiles/fasttext-shared.dir/objects1.rsp 

so I get then the following output:

-rw-r--r--   1 loretoparisi  staff  870544 30 Lug 10:40 libfasttext.a
-rw-r--r--   1 loretoparisi  staff  306646 30 Lug 10:40 fasttext.wasm
-rw-r--r--   1 loretoparisi  wheel  114099 30 Lug 10:40 fasttext.js
-rw-r--r--   1 loretoparisi  staff  870504 30 Lug 10:40 libfasttext_pic.a
-rw-r--r--   1 loretoparisi  staff  663728 30 Lug 10:40 libfasttext.so

This turns out to be a valid javascript file, but it causes an exception when running it

$ node fasttext.js predict-prob /root/lang_id.bin - 2
exception thrown: 5280232 - Exception catching is disabled, this exception cannot be caught. Compile with -s DISABLE_EXCEPTION_CATCHING=0 or DISABLE_EXCEPTION_CATCHING=2 to catch.

while it should get from stdin like

./fasttext predict-prob /root/lang_id.bin - 2
ja das is seher gut heute
__label__deu 0.999483 __label__bar 4.1711e-05

May this error due to stdin in node.js? The command like seems to be fine, since it accepts and parses the parameters in the right way:

$ node fasttext.js predict-prob
usage: fasttext predict[-prob] <model> <test-data> [<k>] [<th>]

  <model>      model filename
  <test-data>  test data filename (if -, read from stdin)
  <k>          (optional; 1 by default) predict top k labels
  <th>         (optional; 0.0 by default) probability threshold

I have then modularized it like emcc -s MODULARIZE=1 -s LEGACY_VM_SUPPORT=1 -s WASM=0 -O1 libfasttext.so -o libfasttext.js, and called within a node module like:

let FastTextModule = require('./fasttext-module.js');
let Module = null;
if (Module) {
    return;
}
Module = {
    noExitRuntime: true,
    noInitialRun: false,
    preInit: [],
    preRun: [],
    postRun: [function () {
        console.log(`Loaded Javascript Module OK`);
    }],
    locateFile: "./",
    arguments: ['predict-prob', '/root/lang_id.bin', '-', 2]
};
Module['locateFile'] = function (path, prefix) {
    return prefix + path;
}
FastTextModule(Module);

But I'm getting the same error: exception thrown: 5278216 - Exception catching is disabled, this exception cannot be caught. Compile with -s DISABLE_EXCEPTION_CATCHING=0 or DISABLE_EXCEPTION_CATCHING=2 to catch.

In the same way I can run it setting noInitialRun:true and then doing

Module.callMain(['predict-prob', '/root/ft_lang_model.bin', 'test.csv', '2']);

but having that error anyways.

loretoparisi
  • 15,724
  • 11
  • 102
  • 146
  • 1
    so `-s DISABLE_EXCEPTION_CATCHING=0` help? – Jiang YD Jul 31 '18 at 09:31
  • Basically it throws out the exception: `exception thrown: 5279128` – loretoparisi Jul 31 '18 at 16:35
  • so I think it means the program run to exception. did you tried the native clang built version firstly? – Jiang YD Aug 01 '18 at 09:46
  • yes it works when using the executable built with `make`, and passing the same parameters as `Module['arguments']` in that order. My wonder is how `emscripten` handles the `stdin` and `stdout`, maybe this could cause that exception. – loretoparisi Aug 01 '18 at 11:04
  • 1
    https://kripken.github.io/emscripten-site/docs/api_reference/Filesystem-API.html – Jiang YD Aug 02 '18 at 02:34
  • @JiangYD thank you, this could be a starting point. Maybe I need more `Module` configuration before `callMain` and run to setup `stdin` and `stdout`. The native clients uses `std::cout` to write to `stdout`. I have submitted a question here as well: https://github.com/kripken/emscripten/issues/6923 – loretoparisi Aug 02 '18 at 08:55

0 Answers0