Use QWebEngineView to Display Something Larger Than 2MB?

Question

I'm trying to display some Plot.ly or Plot.ly Dash plots ( I haven't settled on using one or the other, so I'm experimenting with both right now) in a PyQt5 GUI using QWebEngineView. This doesn't work for any plots larger than 2MB due to some Chromium-level hardcoded restriction.

I found one similar question that is pretty much identical in terms of our needs. It looks like the OP actually found an answer, but unfortunately for me, they didn't post an example of working code or explain what they did to make it work. I do not understand enough of the underlying theory to piece together an answer with the resources linked in this other question, and my Stack reputation isn't high enough to comment and ask the OP what exactly worked.

Here is a minimum reproducible example that displays a plot embedded in the GUI. It's a modification of an answer to a question about embedding Plotly plots in PyQt5 GUIs here:

import numpy as np
import plotly.offline as po
import plotly.graph_objs as go

from PyQt5.QtWebEngineWidgets import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
import sys


def show_qt(fig):
    raw_html = '<html><head><meta charset="utf-8" />'
    raw_html += '<script src="https://cdn.plot.ly/plotly-latest.min.js"></script></head>'
    raw_html += '<body>'
    raw_html += po.plot(fig, include_plotlyjs=False, output_type='div')
    raw_html += '</body></html>'

    fig_view = QWebEngineView()
    # setHtml has a 2MB size limit, need to switch to setUrl on tmp file
    # for large figures.
    fig_view.setHtml(raw_html)
#    fig_view.setUrl(QUrl('temp-plot.html'))
    fig_view.show()
    fig_view.raise_()
    return fig_view


if __name__ == '__main__':
    app = QApplication(sys.argv)

    # Working small plot:
    fig = go.Figure(data=[{'type': 'scattergl', 'y': [2, 1, 3, 1]}])
    # Not working large plot:
#    t = np.arange(0, 200000, 1)
#    y = np.sin(t/20000)
    fig = go.Figure(data=[{'type': 'scattergl', 'y': y}])
#    po.plot(fig)

    fig_view = show_qt(fig)
    sys.exit(app.exec_())

Here is a modified version that demonstrates how a large data set cannot be displayed the same way:

import numpy as np
import plotly.offline as po
import plotly.graph_objs as go

from PyQt5.QtWebEngineWidgets import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
import sys


def show_qt(fig):
    raw_html = '<html><head><meta charset="utf-8" />'
    raw_html += '<script src="https://cdn.plot.ly/plotly-latest.min.js"></script></head>'
    raw_html += '<body>'
    raw_html += po.plot(fig, include_plotlyjs=False, output_type='div')
    raw_html += '</body></html>'

    fig_view = QWebEngineView()
    # setHtml has a 2MB size limit, need to switch to setUrl on tmp file
    # for large figures.
    fig_view.setHtml(raw_html)
#    fig_view.setUrl(QUrl('temp-plot.html'))
    fig_view.show()
    fig_view.raise_()
    return fig_view


if __name__ == '__main__':
    app = QApplication(sys.argv)

    # Working small plot:
#    fig = go.Figure(data=[{'type': 'scattergl', 'y': [2, 1, 3, 1]}])
    # Not working large plot:
    t = np.arange(0, 200000, 1)
    y = np.sin(t/20000)
    fig = go.Figure(data=[{'type': 'scattergl', 'y': y}])
#    po.plot(fig)

    fig_view = show_qt(fig)
    sys.exit(app.exec_())

Lastly, here is something I tried to get the large plot to display with QUrl pointing to a local html plot on the disk:

import numpy as np
import plotly.offline as po
import plotly.graph_objs as go

from PyQt5.QtWebEngineWidgets import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
import sys


def show_qt(fig):
    raw_html = '<html><head><meta charset="utf-8" />'
    raw_html += '<script src="https://cdn.plot.ly/plotly-latest.min.js"></script></head>'
    raw_html += '<body>'
    raw_html += po.plot(fig, include_plotlyjs=False, output_type='div')
    raw_html += '</body></html>'

    fig_view = QWebEngineView()
    # setHtml has a 2MB size limit, need to switch to setUrl on tmp file
    # for large figures.
#    fig_view.setHtml(raw_html)
    fig_view.setUrl(QUrl('temp-plot.html'))
    fig_view.show()
    fig_view.raise_()
    return fig_view


if __name__ == '__main__':
    app = QApplication(sys.argv)

    # Working small plot:
#    fig = go.Figure(data=[{'type': 'scattergl', 'y': [2, 1, 3, 1]}])
    # Not working large plot:
    t = np.arange(0, 200000, 1)
    y = np.sin(t/20000)
    fig = go.Figure(data=[{'type': 'scattergl', 'y': y}])
#    po.plot(fig)

    fig_view = show_qt(fig)
    sys.exit(app.exec_())

The plot was generated with:

import numpy as np
import plotly.offline as po
import plotly.graph_objs as go
t = np.arange(0, 200000, 1)
y = np.sin(t/20000)
fig = go.Figure(data=[{'type': 'scattergl', 'y': y}])
po.plot(fig)

Ok, I edited the post to include working code that illustrates the problem. — squiiidz, Jul 30 '19 at 18:00

eyllanesc · Accepted Answer · 2019-07-31T02:29:18.230

As this answer indicates a possible solution is to use a QWebEngineUrlSchemeHandler, in the next section I have created a class that allows you to register functions that are invoked through custom urls:

qtplotly.py

from PyQt5 import QtCore, QtWebEngineCore, QtWebEngineWidgets

import plotly.offline as po
import plotly.graph_objs as go


class PlotlySchemeHandler(QtWebEngineCore.QWebEngineUrlSchemeHandler):
    def __init__(self, app):
        super().__init__(app)
        self.m_app = app

    def requestStarted(self, request):
        url = request.requestUrl()
        name = url.host()
        if self.m_app.verify_name(name):
            fig = self.m_app.fig_by_name(name)
            if isinstance(fig, go.Figure):
                raw_html = '<html><head><meta charset="utf-8" />'
                raw_html += '<script src="https://cdn.plot.ly/plotly-latest.min.js"></script></head>'
                raw_html += "<body>"
                raw_html += po.plot(fig, include_plotlyjs=False, output_type="div")
                raw_html += "</body></html>"
                buf = QtCore.QBuffer(parent=self)
                request.destroyed.connect(buf.deleteLater)
                buf.open(QtCore.QIODevice.WriteOnly)
                buf.write(raw_html.encode())
                buf.seek(0)
                buf.close()
                request.reply(b"text/html", buf)
                return
        request.fail(QtWebEngineCore.QWebEngineUrlRequestJob.UrlNotFound)


class PlotlyApplication(QtCore.QObject):
    scheme = b"plotly"

    def __init__(self, parent=None):
        super().__init__(parent)
        scheme = QtWebEngineCore.QWebEngineUrlScheme(PlotlyApplication.scheme)
        QtWebEngineCore.QWebEngineUrlScheme.registerScheme(scheme)
        self.m_functions = dict()

    def init_handler(self, profile=None):
        if profile is None:
            profile = QtWebEngineWidgets.QWebEngineProfile.defaultProfile()
        handler = profile.urlSchemeHandler(PlotlyApplication.scheme)
        if handler is not None:
            profile.removeUrlSchemeHandler(handler)

        self.m_handler = PlotlySchemeHandler(self)
        profile.installUrlSchemeHandler(PlotlyApplication.scheme, self.m_handler)

    def verify_name(self, name):
        return name in self.m_functions

    def fig_by_name(self, name):
        return self.m_functions.get(name, lambda: None)()

    def register(self, name):
        def decorator(f):
            self.m_functions[name] = f
            return f

        return decorator

    def create_url(self, name):
        url = QtCore.QUrl()
        url.setScheme(PlotlyApplication.scheme.decode())
        url.setHost(name)
        return url

main.py

import numpy as np
import plotly.graph_objs as go

from PyQt5 import QtCore, QtWidgets, QtWebEngineWidgets

from qtplotly import PlotlyApplication

# PlotlyApplication must be created before the creation
# of QGuiApplication or QApplication
plotly_app = PlotlyApplication()


@plotly_app.register("scatter")
def scatter():
    t = np.arange(0, 200000, 1)
    y = np.sin(t / 20000)
    fig = go.Figure(data=[{"type": "scattergl", "y": y}])
    return fig


@plotly_app.register("scatter2")
def scatter2():
    N = 100000
    r = np.random.uniform(0, 1, N)
    theta = np.random.uniform(0, 2 * np.pi, N)

    fig = go.Figure(
        data=[
            {
                "type": "scattergl",
                "x": r * np.cos(theta),
                "y": r * np.sin(theta),
                "marker": dict(color=np.random.randn(N), colorscale="Viridis"),
            }
        ]
    )
    return fig


@plotly_app.register("scatter3")
def scatter3():
    x0 = np.random.normal(2, 0.45, 30000)
    y0 = np.random.normal(2, 0.45, 30000)

    x1 = np.random.normal(6, 0.4, 20000)
    y1 = np.random.normal(6, 0.4, 20000)

    x2 = np.random.normal(4, 0.3, 20000)
    y2 = np.random.normal(4, 0.3, 20000)

    traces = []
    for x, y in ((x0, y0), (x1, y1), (x2, y2)):
        trace = go.Scatter(x=x, y=y, mode="markers")
        traces.append(trace)

    fig = go.Figure(data=traces)
    return fig


class Widget(QtWidgets.QWidget):
    def __init__(self, parent=None):
        super().__init__(parent)

        self.m_view = QtWebEngineWidgets.QWebEngineView()

        combobox = QtWidgets.QComboBox()
        combobox.currentIndexChanged[str].connect(self.onCurrentIndexChanged)
        combobox.addItems(["scatter", "scatter2", "scatter3"])

        vlay = QtWidgets.QVBoxLayout(self)
        hlay = QtWidgets.QHBoxLayout()
        hlay.addWidget(QtWidgets.QLabel("Select:"))
        hlay.addWidget(combobox)
        vlay.addLayout(hlay)
        vlay.addWidget(self.m_view)
        self.resize(640, 480)

    @QtCore.pyqtSlot(str)
    def onCurrentIndexChanged(self, name):
        self.m_view.load(plotly_app.create_url(name))


if __name__ == "__main__":
    import sys

    app = QtWidgets.QApplication(sys.argv)
    # Init_handler must be invoked before after the creation
    # of QGuiApplication or QApplication
    plotly_app.init_handler()
    w = Widget()
    w.show()
    sys.exit(app.exec_())

Structure:

├── main.py
└── qtplotly.py

Output:

Thanks for the response. I tried running main.py and got the error, "AttributeError: module 'PyQt5.QtWebEngineCore' has no attribute 'QWebEngineUrlScheme'" What version of PyQt5 are you using? I'm using 5.9.2. This is for line 47 of qtplotly.py, scheme = QtWebEngineCore.QWebEngineUrlScheme(PlotlyApplication.scheme) — squiiidz, Jul 31 '19 at 19:01
Nevermind, I switched to a Linux environment from MacOS and it worked fine. Thank you! — squiiidz, Jul 31 '19 at 20:02
If anyone finds this and experiences a similar problem, I think it has something to do with the fact that I'm using Anaconda on my Mac and stock python3 on my linux machine — squiiidz, Jul 31 '19 at 20:32
@squiiidz The problem is the version of PyQt5 you have, QWebEngineUrlScheme was introduced in PyQt5 5.12 — eyllanesc, Jul 31 '19 at 20:46
Thanks, I had a brain-fail moment and thought, "5.9 > 5.12 therefore 5.9.2 is newer than 5.12". Oops. One last question, this time about Plotly: I'm working with the example you posted and noticed, while it *works*, the large scatter2 and scatter3 plots are very laggy when it comes to interacting with them. Is Plotly just not very good at displaying large amounts of data in interactive plots? I'm trying to build a tool to allow rapidly displaying data so people can zoom in and out and pan through it quickly, and this amount of lag precludes that use. — squiiidz, Aug 01 '19 at 14:25
I figured it out (I think): scatter3 uses scatter rather than scattergl. Scattergl was designed for large data sets. If I disable scatter3, then the other two plots exist without lag. — squiiidz, Aug 01 '19 at 19:43
@squiiidz As I pointed out, I am not an expert in plotly, so my intention with the "scaters" was to show you how my implementation works, I didn't want to get on the side of plotly optimization since that depends on you. — eyllanesc, Aug 01 '19 at 19:46
@ellyanesc, oh, I understand and I wasn't trying to criticize you. I just wanted to leave that tidbit of information here for posterity in case someone else runs into this problem and finds this post in the future. I've spent years lurking on Stack and have often wished the original posters would document the results of their questions better, so I'm doing that here. Once again, thank you so much for your help! — squiiidz, Aug 02 '19 at 14:43
Sorry, I have a followup question: The goal of this was to be able to embed plotly plots in tabs of an existing QTabWidget. When I apply the embedding procedure in my example code to put the plotly app in a tab, I get a type error for addWidget, as the plotly application is not a QWidget. Briefly, ```python self.tab5 = QWidget() self.TabsContainer.addTab(self.tab5, "Tab 5") self.tab5.layout = QGridLayout(self) self.SinPlotly = PlotlyApplication() self.tab5.layout.addWidget(self.SinPlotly) self.tab5.setLayout(self.tab5.layout) ``` — squiiidz, Aug 02 '19 at 16:44
@squiiidz PlotlyApplication is not a QWidget, what you have to do is create a QWebEngineView. Analyze my answer better since I have placed certain conditions in my code comments — eyllanesc, Aug 02 '19 at 16:59
@eyllanesc what happen if I want to pass a dataframe to plot it? How should be implemented? — soy_elparce, Jun 08 '22 at 04:19

Use QWebEngineView to Display Something Larger Than 2MB?

1 Answers1

Linked

Related