-1

I'm trying to crawl all data of the products of a webpage but I am stuck at the "view-more" div, I need to use a crawler to click on it to view all products, I tried to click it sometimes, and use the full product URL to crawl but it still not work.

-this is the "view-more" class:

<div class="view-more">
 <a href="javascript:;">Xem thêm <span class="remain">150</span> Điện thoại</a>

when there is no more product to view, the style of "view-more" class becomes "display: none"

        $url = "https://www.thegioididong.com/dtdd";

    $client = new Client();

    $crawler = $client->request('GET', $url);
    $crawler->filter('#categoryPage > div.container-productbox')->reduce(
        function(Crawler $node){
            $test = $node->filter('div.view-more')->attr('style');
            while($test == null){
                // here I want to click on the <a> element
                //$node->filter('div.view-more > a')->????;
            }
            echo $test;
        }
    );

this is the first time I ask a question here, pls don't judge me :)

Nayana Chandran
  • 1,416
  • 1
  • 16
  • 30
duclong
  • 11
  • 2

1 Answers1

0

no you cant click via PHP. But there are two options: Option a: the content is already loaded and readable in pagesource. Option b: content is missing and on click event a new request gets sended. You can send this request manually via php.

CampingCow
  • 120
  • 10