0

I have built a web scraper, but I have to manually add the URL and class it wants to scrape to the source code. I want to automate this by adding a front-end. So I have a function connected to a form that on submit edits the URL and a class variable.

I then want to use that variable in the cheerio command. I'm not entirely sure how to do this. It uses Express, Axios, and Cheerio.

const PORT = 8000;
const axios = require('axios');
const cheerio = require('cheerio');
const express = require('express');
const app = express();
let url = "";
let classes = "";

document.getElementById("button-submit").addEventListener("submit", onSubmitFunction);

const onSubmitFunction = () => {

    url = document.getElementById("url-input");
    classes = document.getElementById("class-input");
}


axios(url)
    .then(response => {
        const html = response.data;
        const $ = cheerio.load(html);

        const listing = [];

        $('.number', html).each(function() {
            const room = $(this).text()
            listing.push({
                room
            })
        })

        let text = "";
        listing.forEach(myFunction);
        document.getElementById("text-output").innerHTML = text;
 
        function myFunction(item, index) {
            text += index + ": " + item + "<br>"; 
        }

     })

    <div class="m-4">
        <label for="exampleFormControlInput1" class="form-label">URL</label>
        <input type="text" class="form-control" id="url-input" placeholder="Enter URL Here.">
    </div>
    <div class="m-4">
        <label for="exampleFormControlInput2" class="form-label">CLASS</label>
        <input type="text" class="form-control" id="class-input" placeholder="Enter Class Here.">
    </div>
    <div class="m-4">
        <label for="exampleFormControlTextarea1" class="form-label">OUTPUT</label>
        <textarea class="form-control" id="text-output" rows="3"></textarea>
    </div>
    <input class="btn btn-primary m-4" id="button-submit" type="submit" value="Submit">

I want the class variable that is set dynamically by the front-end to be inserted into where the .number piece of code. I'm just not entirely sure what the correct syntax is to do that.

ggorlen
  • 44,755
  • 7
  • 76
  • 106
Militant.Rectangle
  • 3
  • 4
  • How are you using express and document.querySelector in the same app? I'm confused about whether this is running on the server or the client. [What are you trying to achieve exactly](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem/233676#233676)? – ggorlen Aug 24 '22 at 01:22
  • I'm trying to connect an HTML front-end to the back-end. So I can fill out a form that takes and URL and a class sends it to the back-end that does the scraping, and send it back to a text area on the front end. – Militant.Rectangle Aug 24 '22 at 08:52
  • Probably boils down to the same misunderstanding as [Unable to Manipulate DOM with cheerio on nodejs](https://stackoverflow.com/questions/31677151/unable-to-manipulate-dom-with-cheerio-on-nodejs). – ggorlen Jan 02 '23 at 19:07

0 Answers0