-3

Summary :

I want to fetch all Play store search result, problems is that, Apps that show after scroll that are not show in PHP file_get_content().

Detail:

I'm trying to make a php based play store scraper.

I checked all stackoverflow answers and github example, but they all are old, and not working. This is because previously Play store use "start" parameter for more apps/Next page.. Now play store shows more apps on scroll

So after so many research, I decide to make my own scraper.

What I am doing:

file_get_contents() : to fetch the query url from play store like so:

$result = file_get_content( "https://play.google.com/store/search?q=football" );

This only returns 20 apps in the result, I want 250.

Play store shows only 20 apps on the query result, more apps are shown ONLY on scrolling.

I try to get google ajax URL from "network tab", But google use token parameter for next scroll page...

Questions:

How can I get scrolled content with PHP? If not possible, how can I get play store all search result?

Otherwise how can I fetch page content with PHP that shows with javascript?

Tim Ogilvy
  • 1,923
  • 1
  • 24
  • 36
asad app
  • 321
  • 1
  • 2
  • 14

1 Answers1

0

Have a look at the store site in chrome dev tools. It has a perfect data source, but you may have trouble getting the tokens. I doubt they want this scraped.

General:

Request URL: https://play.google.com/store/apps?authuser=0
Request Method: POST
Status Code: 200
Remote Address: 216.58.220.142:443

Form Data:

start: 20
num: 5
numChildren: 10
pagTok: CBQQFA==:S:ANO1ljJ62NU
ipf: 1
xhr: 1
token: OmILlMEkRv_FCJ1tveWI8rtELHQ:1459318564260

Tim Ogilvy
  • 1,923
  • 1
  • 24
  • 36
  • start:20 ??, in my chrome and mozila its show "start:0, num:0.".. i found that, these parameter are not working, play store using token for next page. – asad app Mar 30 '16 at 06:59