Does anyone know if I can download Wikipedia text data with the navigation popups included?
I know there are existing text dump datasets [1,2], but as far as I can tell, they do not include the navigation popup text.
[1] https://dumps.wikimedia.org/backup-index.html [2] https://www.kiwix.org/en/download/
If you want to download the text inside the navigation popup, you can use the following URL:
https://en.wikipedia.org/api/rest_v1/page/summary/<search_term>
where <search_term> is the term you want to use.
Example about the summary of Colombia - shown in the navigation popup window:
https://en.wikipedia.org/api/rest_v1/page/summary/Colombia
Result:
{"type":"standard","title":"Colombia","displaytitle":"<span class=\"mw-page-title-main\">Colombia</span>","namespace":{"id":0,"text":""},"wikibase_item":"Q739","titles":{"canonical":"Colombia","normalized":"Colombia","display":"<span class=\"mw-page-title-main\">Colombia</span>"},"pageid":5222,"thumbnail":{"source":"https://upload.wikimedia.org/wikipedia/commons/thumb/2/21/Flag_of_Colombia.svg/320px-Flag_of_Colombia.svg.png","width":320,"height":213},"originalimage":{"source":"https://upload.wikimedia.org/wikipedia/commons/thumb/2/21/Flag_of_Colombia.svg/900px-Flag_of_Colombia.svg.png","width":900,"height":600},"lang":"en","dir":"ltr","revision":"1130831378","tid":"2a3b2e60-89de-11ed-8f85-3d0f6ce6a5cb","timestamp":"2023-01-01T05:02:29Z","description":"Country in South America","description_source":"local","coordinates":{"lat":4,"lon":-72},"content_urls":{"desktop":{"page":"https://en.wikipedia.org/wiki/Colombia","revisions":"https://en.wikipedia.org/wiki/Colombia?action=history","edit":"https://en.wikipedia.org/wiki/Colombia?action=edit","talk":"https://en.wikipedia.org/wiki/Talk:Colombia"},"mobile":{"page":"https://en.m.wikipedia.org/wiki/Colombia","revisions":"https://en.m.wikipedia.org/wiki/Special:History/Colombia","edit":"https://en.m.wikipedia.org/wiki/Colombia?action=edit","talk":"https://en.m.wikipedia.org/wiki/Talk:Colombia"}},"extract":"Colombia, officially the Republic of Colombia, is a country in South America with insular regions in North America—near Nicaragua's Caribbean coast—as well as in the Pacific Ocean. The Colombian mainland is bordered by the Caribbean Sea to the north, Venezuela to the east and northeast, Brazil to the southeast, Ecuador and Peru to the south and southwest, the Pacific Ocean to the west, and Panama to the northwest. Colombia is divided into 32 departments and the Capital District of Bogotá, the country's largest city. It covers an area of 1,141,748 square kilometers, and has a population of 52 million. Colombia's cultural heritage—including language, religion, cuisine, and art—reflects its history as a Spanish colony, fusing cultural elements brought by immigration from Europe and the Middle East, with those brought by enslaved Africans, as well as with those of the various Indigenous civilizations that predate colonization. Spanish is the official state language, although English and 64 other languages are recognized regional languages.","extract_html":"<p><b>Colombia</b>, officially the <b>Republic of Colombia</b>, is a country in South America with insular regions in North America—near Nicaragua's Caribbean coast—as well as in the Pacific Ocean. The Colombian mainland is bordered by the Caribbean Sea to the north, Venezuela to the east and northeast, Brazil to the southeast, Ecuador and Peru to the south and southwest, the Pacific Ocean to the west, and Panama to the northwest. Colombia is divided into 32 departments and the Capital District of Bogotá, the country's largest city. It covers an area of 1,141,748 square kilometers, and has a population of 52 million. Colombia's cultural heritage—including language, religion, cuisine, and art—reflects its history as a Spanish colony, fusing cultural elements brought by immigration from Europe and the Middle East, with those brought by enslaved Africans, as well as with those of the various Indigenous civilizations that predate colonization. Spanish is the official state language, although English and 64 other languages are recognized regional languages.</p>"}
