how do parsing DynamicWebsite data

Question

Iam Daniel und totaly new in Perl.

My Problem is parsing a dynamic website.

I have a local website that is filled dynamically. my perl script only loads the unfilled version

> `<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" dir="ltr"> 
    <head>
        <title>About</title>
        <meta http-equiv="Content-Type" content="text/html;charset=UTF-8"/>
        <meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=0"/>
        <!-- Load the 'handheld' style sheet by default -->
        <link type="text/css" rel="stylesheet" href="handheld.css" id="stylesheetused"/>
        <script type="text/javascript">
            if (((window.devicePixelRatio <= 1.0) && (screen.width >= 801)) || (navigator.userAgent.indexOf("MSIE") !== -1) ) {
                document.getElementById('stylesheetused').href='desktop.css';
        }
        </script>
        <script type="text/javascript" src="jquery.js"></script>
        <script type="text/javascript" src="ServerInterface.js"></script>
        <script type="text/javascript" src="HeaderFooter.js"></script>
        <script type="text/javascript" src="About.js"></script>
    </head>

    <body>
        <!-- Import the shared page header. -->
        <div id="shared-header"></div>
        <script type="text/javascript">
            $("#shared-header").load("header.html", function (){
                serverInterface.getFactoryConfig(headerFooter.headerCallback);
            });
        </script>

        <!-- The main configuration form. -->
        <div class="config-form">
            <div class="config-title">
                Hardware
            </div>
            <!-- Display the product name. -->
            <div class="config-element-top">
                <span id="display-inline" class="config-name">Name:</span>
                <span id="product-type" class="read-only-text">Not available</span>
            </div>
            <!-- Display the product revision. -->
            <div class="config-element-middle">
                <span id="display-inline" class="config-name">Revision:</span>
                <span id="product-rev" class="read-only-text">Not available</span>
            </div>`

The "Not available" Parts where filled dynamicly with js. the first "Not available" is what I need.

I have a perl script that finds the first "Not available" but i need the filled one.

<span id="product-type" class="read-only-text">Denon AVR-X4500H</span>

my script

#!/usr/bin/perl -w
use LWP::Simple;
use HTML::TreeBuilder::XPath;
use Data::Dumper;
use strict;
my $debug=1;

my $page = get 'http://192.168.1.65/settings/about.html' or die $!;  
my $p = HTML::TreeBuilder::XPath->new_from_content( $page );
binmode( STDOUT, ':utf8');

my @names= $p->findnodes( '//div[@class="config-element-top"]')->[0];
foreach my $name (@names){
        my $title = $name->findvalue( './/span[@id="product-type"]');
        print "Model: $title\n";

}

The Output is

Model: Not available

how can I load the filled version?

regards

I don't think `LWP::Simple` supports JavaScript. Maybe try [`WWW::Mechanize::Firefox`](https://metacpan.org/pod/WWW::Mechanize::Firefox) instead? See e.g. [this](https://stackoverflow.com/a/25130317/2173773) post. — Håkon Hægland, Mar 01 '20 at 17:16
Since Perl 5.6 was released twenty years ago, it's been best practice to have `use warnings` instead of `-w` on the shebang. — Dave Cross, Mar 02 '20 at 09:16

how do parsing DynamicWebsite data

0 Answers0