When I request data off a website https://unqnft.io/#/market or https://unqnft.io/#/trades, in the text there is a ton of content missing. The page seems to be fetching the data from elsewhere.
use scraper::{Html, Selector};
error_chain! {
foreign_links {
ReqError(reqwest::Error);
IoError(std::io::Error);
}
}
#[tokio::main]
async fn main() -> Result<()> {
println!("Start of the file: ");
let res = reqwest::get("https://unqnft.io/#/market")
.await?
.text()
.await?;
println!("Length of stories {:?} ", res);
let fragment = Html::parse_document(&res);
// Additionally, how would i select something that has a tag of #anch_10?
let stories = Selector::parse("#anch_10").unwrap();
println!(" ");
println!(" ");
println!(" ");
for story in fragment.select(&stories) {
let story_txt = story.text().collect::<Vec<_>>();
println!("{:?}", story_txt);
}
This returns This:
"<!doctype html><html lang=\"en\"><head><meta charset=\"utf-8\"><meta name=\"theme-color\" content=\"#000000\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"><link rel=\"manifest\" href=\"manifest.json\"><title>Unique Network NFT Marketplace</title><link rel=\"preconnect\" href=\"https://fonts.gstatic.com\"><link href=\"https://fonts.googleapis.com/css2?family=Roboto:wght@400;500;600;700&display=swap\" rel=\"stylesheet\"><link href=\"https://fonts.googleapis.com/css2?family=Raleway:wght@500&display=swap\" rel=\"stylesheet\"><link href=\"https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap\" rel=\"stylesheet\"><link rel=\"stylesheet\" href=\"theme.css\"><link href=\"react.01.95688c15.css\" rel=\"stylesheet\"><link href=\"main.2c2dad92.css\" rel=\"stylesheet\"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><script>var link = document.querySelector(\"link[rel~='icon']\");\n if (!link) {\n link = document.createElement('link');\n link.rel = 'icon';\n document.getElementsByTagName('head')[0].appendChild(link);\n }\n link.href = 'favicon.ico';</script><script src=\"./env.js\"></script><div id=\"root\"></div><div id=\"tooltips\"></div><script>if (window.self !== window.top) {\n window.top.location.href = window.location.href;\n }</script><script src=\"other.00.43aa4f28.js\"></script><script src=\"robohash.02.16a88645.js\"></script><script src=\"polkadot.01.21109aeb.js\"></script><script src=\"robohash.00.3a841187.js\"></script><script src=\"polkadot.02.e99664ce.js\"></script><script src=\"react.01.69a27675.js\"></script><script src=\"robohash.01.dbfaecb9.js\"></script><script src=\"other.02.a8d8c091.js\"></script><script src=\"polkadot.00.31bbf1a4.js\"></script><script src=\"react.00.cb68007d.js\"></script><script src=\"other.01.733fa76f.js\"></script><script src=\"main.5af7eb1e.js\"></script></body></html>"
and inspecting the elements i can trace it to somewhere in 'App-container'. How do i acceess that? They seem to be behind theese "Robotshash" Scripts that i don't see.
<script src=\"other.00.43aa4f28.js\"></script><script src=\"robohash.02.16a88645.js\"></script><script src=\"polkadot.01.21109aeb.js\"></script><script src=\"robohash.00.3a841187.js\"></script><script src=\"polkadot.02.e99664ce.js\"></script><script src=\"react.01.69a27675.js\"></script><script src=\"robohash.01.dbfaecb9.js\"></script><script src=\"other.02.a8d8c091.js\"></script><script src=\"polkadot.00.31bbf1a4.js\"></script><script src=\"react.00.cb68007d.js\"></script><script src=\"other.01.733fa76f.js\"></script><script src=\"main.5af7eb1e.js\"></script></body></html>
