0

I have tried scraping a web page with ruby mechanize but is not working. Basically that website have some products and i need the links of the products.

HTML

1

I was test this code below, and I expected the links to the products but the output doesn't show anything. `

require 'mechanize'

agent = Mechanize.new
page = agent.get('https://shopee.com.br/Casa-e-Decora%C3%A7%C3%A3o-cat.11059983/')


page.css('col-xs-2-4 shopee-search-item-result__item').each do |product|
  puts product.link.text
end

`

vimuth
  • 5,064
  • 33
  • 79
  • 116
Lucas Cunha
  • 1
  • 2

1 Answers1

1

Part of the page you are trying to parse is being rendered client side, so when mechanize gets the HTML it does not contain the links you are looking for.

Luckily for you the website is using a JSON API so it is pretty easy to extract the information of the products.

javiyu
  • 1,336
  • 6
  • 9