I'm trying to get all the page titles in Wikipedia in namespace using the API as following:
https://en.wikipedia.org/w/api.php?action=query&format=xml&list=allpages&apnamespace=0&apfilterredir=nonredirects&aplimit=max&continue=-||&apcontinue=BASE_PAGE_TITLE
I keep requesting this url and checking the response if contains continue tag. if yes, then I use same request but change the BASE_PAGE_TITLE to the value in apcontinue attribute in the response.
My applications had been running since 3 days and number of retrieved exceeds 30M, whereas it is about 13M in the dumps.
any idea?