extract information from a website using Qt?

Question

I'd like extract the information in the b tag => 123456789

this is the HTML source :

<body>
       <div>
          <table>
               <tbody>
                     <tr>
                         <td class="myclass">
                               <b>123456789</b>
                         </td>
                     </tr>
              </tbody>
          </table>
       </div>
 </body>

So, I tried this :

void My_Test_Dialog::on_pushButton_clicked()
{


        QWebView *webview = new QWebView(parentWidget());

        webview->load(QUrl("http://www.esesese.com"));

        webview->show();

         // get HTML element information
        QWebElementCollection colls = webview->page()->mainFrame()->findAllElements("td.myclass b");



         foreach(QWebElement elemento, colls)
        {
                    ui->lineEdit_data->setText(elemento.toInnerXml());
        }
}

I have a form with a Button(call update) and a LineEdit, so if I click on the update button, the LineEdit should set the text 123456789 automatically. But my code doesn't work. The Text of the LineEdit remains empty.

I include this :

#include <QtWebKit>
#include <QtWebKitWidgets/QWebFrame>
#include <QWebView>

QT file.pro is :

QT += core gui
QT += network
QT += webkit
QT += webkitwidgets

Consider giving the webview time to load the page to completion before you can click the "update" button — mrmoje, Sep 08 '13 at 20:15
No, you don't. At least this is what your code says. You load your website in your on_pushButton_clicked slot. In the same slot you try to evaluate the result. Instant download? Use QWebView's loadFinished signal to detemine when your url load is finished. **Then** you can try to evaluate the content. — Greenflow, Sep 08 '13 at 22:14
and , how can i use the loadFinished() ? could you make an example — NPLS, Sep 09 '13 at 09:22

score 1 · Answer 1 · answered Apr 18 '15 at 18:22

As already mentioned you need to make sure that you wait long enough for the data in your QWebView to load.

You can do this (very simplistic) with something like this:

Define the webView as part of dialog class and also declare a slot that you can later connect to the signal of the web view

class My_Test_Dialog
{
public slots:

  // slot to read your data once you are finished
  void readPage(bool ok);

  // whatever else you did
private: 
  QWebView *webView;

}

Then e.g. in the constructor or elsewhere you can create the webView and connect its loadFinished() signal to the readPage() slot shown also in the class definition above

// create QWebview and connect its loadFinished signal to our slot 
webView = new QWebView(this);
QObject::connect(webView,SIGNAL(loadFinished(bool)), this, SLOT( readPage(bool) ) );

then in your on_pushButton_clicked() method you only load the page (and show the webview if that is what you need)

void My_Test_Dialog::on_pushButton_clicked()
{
  webView->load(QUrl("http://www.esesese.com"));
}

and then once the dialog finished loading the slot readData() will be automatically called and there you can simply do your read operation

void MyDialog::readPage(bool ok)
{
  // get HTML element information                                                                                                                                                                    
  QWebElementCollection colls = webView->page()->mainFrame()->findAllElements("td.myclass b");

  foreach(QWebElement elemento, colls)
    {
      lineEdit->setText(elemento.toInnerXml());
    }

}

Let me know if this helps.

extract information from a website using Qt?

1 Answers1