0

I've started a project that involves scraping to get some data and part of that functionality includes submitting a URL and scraping that page. Is this something that I'm able to do with Cheerio/Next.js? I've never done web scraping before so I'm trying to learn as I go and was curious about this...

Is it possible to update this URL with an on submit function? Am happy to provide more of the function if need be. Thanks.

  let res = await axios.get(
    'http://www.allrecipes.com/recipe/266826/air-fryer-potato-wedges/'
  );
Gore Sometimes
  • 69
  • 7
  • Cheerio is just a static HTML parser. If you want to interact with the page look into Puppeteer or Playwright. – ggorlen Sep 06 '22 at 19:23
  • Does this answer your question? [How to fix '$(...).click is not a function' in Node/Cheerio](https://stackoverflow.com/questions/56675374/how-to-fix-click-is-not-a-function-in-node-cheerio) – ggorlen Jan 04 '23 at 19:25
  • @ggorlen thank you for sharing that post. Silly question, but I'm asking because I'm honestly not sure cause scraping is new to me, in that post it mentions that you can't interact with jQuery/DOM elements, does that mean that on the page itself or the jQuery that I'm using to select those elements? The scraper itself works and is grabbing what I need in the console. – Gore Sometimes Jan 05 '23 at 20:47
  • Cheerio takes a string of HTML in, transforms it into a tree of objects and lets you traverse and manipulate those objects. After axios delivers a string of HTML to Cheerio, all interaction with the original page is completely absent. You'd have to send a new HTTP request with axios to have another interaction, and if that requires a cookie or JS running in the browser, then it probably can't perform the form submission. Hence the motivation for browser automation. The common mistake is assuming the axios/cheerio combo is somehow related to browser automation. – ggorlen Jan 05 '23 at 20:52

0 Answers0