1

This works on other pages including this sites homepage, but I cannot use it on any pages I want which is that website's items they have for sale. I'm trying to get the price, shipping cost, title, description and other details. I'm using symfony\domcrawler\crawler and symfony\cssSelector but it will not work with the page I am trying to scrape. Is there something else I am not doing or what's the problem i've tried request and goute and it still won't work, I am stuck. There is something going on here I do not know.

Page i'm scraping https://app.cjdropshipping.com/product-detail.html?id=189F56FD-0D65-42C7-88A6-093FA8D7ADB0&push_id=&fromType=

The text i am getting in chrome tools from selector

    require "vendor/autoload.php";

    require "vendor/rmccue/requests/library/Requests.php";
    ///require "vendor/rmccue/requests/library/Requests";
    use Symfony\Component\DomCrawler\Crawler;
    use Symfony\Component\CssSelector\CssSelectorConverter;
    use Symfony\Component\CssSelector;

    $url = 'https://app.cjdropshipping.com/product-detail.html?id=189F56FD-0D65-42C7-88A6-093FA8D7ADB0&push_id=&fromType=';
    $headers = array('Content-Type' => 'application/json');
    $data = array('price' => '.pd-price-span1');
    $response = Requests::post($url, $headers, json_encode($data));
    //echo $response->p->text;

    $crawler = new Crawler($response->body);
    $filter = '#merch-price';
    $catsHTML = $crawler
                ->filter($filter)
                ->each(function (Crawler $node) {
                    return $node->html();
                });
    var_dump($catsHTML);
    //var_dump( $catsHTML);
    //var_dump($response->body);
extrude
  • 11
  • 1
  • Page is being rendered in js (try disabling it in your browser and accessing the page), neither Goute or Crawler support it, you could try using Panther. – msg Apr 07 '20 at 15:15
  • I tried it with javascript disabled still can't get it to work – extrude Jun 28 '20 at 22:13

0 Answers0