x-ray is a web scraping library for Node.js. It can use various pluggable drivers such as HTTP or PhantomJS.
Resources:
Questions tagged [x-ray]
107 questions
0
votes
1 answer
x-ray.js grabbing class attributes and returning it
I am using x-ray scraping tool to scrape some fun sites but it seem that I can't figure out how I can grab the class attributes.
Also, do you guys know the good cheatsheet for this module?
DevBear15
- 219
- 4
- 15
0
votes
2 answers
How do i use x-ray on a meteor app?
I'm trying to use x-ray on Meteor but so far with no luck.
Here's the example I'm testing (it works fine on a basic node app)
import Xray from 'x-ray';
var xray = new Xray();
xray('http://reddit.com/r/meteor/', '.title',
[{
title: '',
href:…
Gonzalo Gutierrez
- 106
- 7
0
votes
1 answer
Populate an array with meta data gathered from Xray
I've been trying to fill an array with metadata that I collect with Xray, and haven't had any success. The function is called by an API route on my server and gets the links from my application.
I seem to be struggling with promises as it takes time…
David Meents
- 621
- 4
- 12
0
votes
1 answer
How to specify language for x-ray npm webscraping?
Subject of the issue
E.g. i want to scrape a book page from play market. Google market is available in multiple languages without url-specific markers. So, how can i scrape the specific version with language i need instead of random…
stkvtflw
- 12,092
- 26
- 78
- 155
0
votes
2 answers
Two div with the same class name, how can I scrape only the first class instance?
var Xray = require('x-ray');
var x = Xray();
var a = false;
var url = "abcdeabcde";
x(url, '.listing_risultati_prodotti .smcc-listing-risultati-prodotto', [{
title: '.first-col a',
link:…
enF
- 59
- 1
- 8
0
votes
1 answer
NodeJS Xray can't crawl to multiple sites to scrape data
So, I need to scrape real estate ads into a nidax.json file. I go to the all ads page, and use the link to the individual ads to take the data I need. I am using NodeJS Xray scraper, but for some reason it does not work.
Sometimes it returns…
Vedran Korponajić
- 47
- 2
- 9
0
votes
1 answer
Write after end error using x-ray
I'm using x-ray library in my MEAN.js project and I can't fix this error message:
Error: write after end
at writeAfterEnd (_stream_writable.js:166:12)
at Writable.write (_stream_writable.js:211:5)
at Writable.end…
Rober
- 726
- 8
- 27
0
votes
0 answers
Different response length on public IP than on local
I streaming JSON to browser using x-ray scraper:
app.get('/scrape', function (req, res) {
// other code
var stream = x(siteUrl, site.item, site.params)
.paginate(site.paginate)
.stream();
stream.pipe(res);
// other…
stmn
- 439
- 1
- 5
- 8
0
votes
2 answers
NodeJS x-ray web-scraper:multiple urls in a loop callback
im using x-ray, which is great but lack of tutorial. anyway, I use an array of urls named urls. In the loop ,each url fetched and return result through callback. in the callback function i need to know what was the url which was parsed. How can i…
dang
- 1,549
- 1
- 20
- 25
0
votes
1 answer
Empty object x-ray scraper in node.js not working
I want to get the first link of the result of here http://search.azlyrics.com/search.php?q=tired+adele
using x-ray node.js scraper. And my code as below :
x('http://search.azlyrics.com/search.php?q=tired+adele', {
link: x('body >…
Cody Jonas
- 151
- 6
0
votes
2 answers
x-ray for grabbing html info from many urls with "for" loop are letting objs undefined
I'm trying to grab info from many urls automatic. I have this array with address called arrayDep and I have a "for" looping my array and entering the web sites. After that, I use x-ray to grab the info I want. At moment I'm using console.log to see…
michelpm1
- 177
- 1
- 1
- 12
0
votes
1 answer
Callback/Promises Implementation in Express.js
I'm writing an Express.js application using x-ray as a scraper to get some info.
I want to create a model for each website I'm scraping (different website = different data/procedures to scrape).
Here's the code of the module:
module.exports.getData…
enF
- 59
- 1
- 8
0
votes
1 answer
Inconsistent web scrape results between node-osmosis, x-ray, and Chrome console
Recently I found node-osmosis is a relatively new module but has powerful features, such as accepting both CSS and XPath selectors, rapid scraping, and nice syntax.
So I made a comparison between node-osmosis and x-ray by running some scrapes using…
pateheo
- 430
- 1
- 5
- 13
0
votes
1 answer
X-Ray scraping and present json in the server browser
I'm using x-ray to scrape a website, however i can't seem to show the correct JSON output in the browser. It works fine, when i write a new json doc like write('result.json') however now when i try to send it to the browser. i'm at the moment using…
Peter Pik
- 11,023
- 19
- 84
- 142
-1
votes
1 answer
keep original values when saving matrix to image in Matlab
Let me explain.
My program takes an x-ray in a format of the x-ray detector ".his" which goes from 0 to 65535, and from those values it can tell you how much of a certain material is in each pixel "4 cm of aluminum" for example.
It does that for…
Fernando Franco Félix
- 139
- 1
- 5