CEFPython window closed without warning when transfering large data

Question

I am trying to create this python binding to a electronjs app I create, where the python part is used to analyze data, and the result (a python dictionary) is passed to the web browser to be visualized. It all worked like a charm for small amount of data, but when I tried to pass a large result dictionary (~200MB), the window was created, but closed without warning in the terminal (I couldn't check devtool), although there seem to be a subprocess kept running on the backend. This problem occurred on both windows and Ubuntu machines, and I would really appreciate your help.

here is my code:

def view(data):

config = data

settings = {
    "debug": True,
    "log_severity": cef.LOGSEVERITY_INFO,
    "log_file": "debug.log",
}
cef.Initialize(settings=settings)
browser_setting = { "file_access_from_file_urls_allowed":True,\
                "universal_access_from_file_urls_allowed": True,\
                "web_security_disabled":True}
browser = cef.CreateBrowserSync(url='file://' + os.path.realpath("index_cefpython.html"),
                                window_title="Javascript Bindings", settings = browser_setting)
browser.SetClientHandler(LoadHandler(config))
bindings = cef.JavascriptBindings()
browser.SetJavascriptBindings(bindings)
cef.MessageLoop()
del browser
cef.Shutdown()

return 


class LoadHandler(object):

def __init__(self, config):
    self.config = config
def OnLoadEnd(self, browser, **_):
    browser.ExecuteFunction("defineData", self.config)

on the JS side, I have:

<script type="text/javascript">
    function defineData(datainput){             

    console.log("start")
    data = datainput;

    Main();
    }
</script>

and here is all the messages printed out in the terminal:

[0312/104311.439:INFO:cefpython_app.cpp(199)] [Browser process] OnBeforeChildProcessLaunch() command line: "<>\Anaconda3\lib\site-packages\cefpython3\subprocess" --type=gpu-process --no-sandbox --locales-dir-path="<>\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path="<>\Anaconda3\lib\site-packages\cefpython3" --lang=en-US --disable-gpu-shader-disk-cache /prefetch:2

[0312/104311.440:INFO:cef_log.cpp(8)] [Browser process] Switch already set, ignoring: disable-gpu-shader-disk-cache

[0312/104311.440:INFO:cefpython_app.cpp(199)] [Browser process] OnBeforeChildProcessLaunch() command line: "<>\Anaconda3\lib\site-packages\cefpython3\subprocess" --type=gpu-process --no-sandbox --locales-dir-path="C:\Users\Xiangyun\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path=" <>\Anaconda3\lib\site-packages\cefpython3" --lang=en-US --disable-gpu-shader-disk-cache --gpu-preferences=KAAAAAAAAACAA4CAAQAAAAAAAAAAAGAAEAAAAAAAAAAAAAAAAAAAACgAAAAEAAAAIAAAAAAAAAAoAAAAAAAAADAAAAAAAAAAOAAAAAAAAAAQAAAAAAAAAAAAAAAKAAAAEAAAAAAAAAAAAAAACwAAABAAAAAAAAAAAQAAAAoAAAAQAAAAAAAAAAEAAAALAAAA --locales-dir-path="<>\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path="<>\Anaconda3\lib\site-packages\cefpython3" --lang=en-US /prefetch:2

[0312/104311.473:INFO:cef_log.cpp(8)] [Browser process] CreateBrowserSync() called

DevTools listening on ws://127.0.0.1:59232/devtools/browser/abaff316-3b5c-4647-8af3-c7c521146d08 [0312/104311.473:INFO:cef_log.cpp(8)] [Browser process] navigateUrl: file://///<>/index_cefpython.html

[0312/104311.475:INFO:cef_log.cpp(8)] [Browser process] CefBrowser::CreateBrowserSync()

[0312/104311.493:INFO:cef_log.cpp(8)] [Browser process] GetPyBrowser(): create new PyBrowser, browserId=1

[0312/104311.516:INFO:cefpython_app.cpp(199)] [Browser process] OnBeforeChildProcessLaunch() command line: "<>\Anaconda3\lib\site-packages\cefpython3\subprocess" --type=renderer --no-sandbox --service-pipe-token=C31AF08C64F1883299C21D068EF5263C --lang=en-US --locales-dir-path=<>\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path="<>\Anaconda3\lib\site-packages\cefpython3" --disable-gpu-shader-disk-cache /prefetch:1

[0312/104311.528:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): underlying frame does not yet exist: browserId = 1, frameId = -4

[0312/104311.529:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): underlying frame does not yet exist: browserId = 1, frameId = -4

[0312/104311.531:INFO:cef_log.cpp(8)] [Browser process] CefBrowser::CreateBrowserSync() succeeded

[0312/104311.532:INFO:cef_log.cpp(8)] [Browser process] CefBrowser window handle = 595526

[0312/104311.533:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): underlying frame does not yet exist: browserId = 1, frameId = -4

[0312/104311.534:INFO:cef_log.cpp(8)] [Browser process] SendProcessMessage(): message=DoJavascriptBindings, arguments size=1

[0312/104311.534:INFO:cef_log.cpp(8)] [Browser process] MessageLoop()

[0312/104311.612:INFO:client_handler.cpp(40)] [Browser process] OnProcessMessageReceived(): OnContextCreated

[0312/104311.613:INFO:cef_log.cpp(8)] [Browser process] V8ContextHandler_OnContextCreated()

[0312/104311.614:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): create new PyFrame, frameId=2

Czarek Tomczak · Accepted Answer · 2019-03-12T16:04:26.173

Javascript bindinsgs should not be used to transfer large data. To send big data use http requests instead. Basically either use AJAX requests and run an internal in-app web server or use Resource Handler which allows to handle requests without running a web server.

See the Tutorial document > "Javascript integration" > "Communication using http requests" section (scroll down a bit after clicking link):

https://github.com/cztomczak/cefpython/blob/master/docs/Tutorial.md#javascript-integration

You could try to send data in smaller parts using javacript bindings, but using requests will be better performant.

Check how your app memory spikes when sending the data. Use process monitor on Linux and process manager on Windows.

To find out what failed when sending 200 MB of data, debug app to get stack trace. If using Linux then download libcef.so with debug symbols from Releases page tagged e.g. v66-upstream. Replace original libcef.so in the cefpython3 package with that one. GDB commands can be found here:

https://github.com/cztomczak/cefpython/blob/master/docs/Knowledge-Base.md#python-crashes-with-segmentation-fault---how-to-debug

Thank you very much for your timely response! by "Communication using http requests", does that mean I have to write the object into json file, set up a server for CEFpython and load the file via AJAX call in the browser? Since I'm just trying to pass a python dictionary to a JS object in the browser (Only once in the beginning), is there a easier/more efficient way to achieve this? Sorry I'm very new to this technology.. — lxiangyun93, Mar 20 '19 at 20:56

CEFPython window closed without warning when transfering large data

1 Answers1