x-ray is a web scraping library for Node.js. It can use various pluggable drivers such as HTTP or PhantomJS.
Resources:
Questions tagged [x-ray]
107 questions
1
vote
1 answer
dynamic links in nodejs/ cheerio/ x-ray
Here is what I am trying to accomplish. I am able to scrape a web page successfully and then extract the information that I need and I have already run this on a couple of websites where the pagination links are readily available in the href…
johnanish
- 91
- 1
- 6
1
vote
2 answers
Select sibling dom element based on the text content of one of sibling
I have following html structure -
Phone Number:
(123) 456-7869
WitVault
- 23,445
- 19
- 103
- 133
1
vote
1 answer
scraping items with x-ray that don't have a single root
I'm running into trouble with scraping items that don't have a single root. Something that is necessary I believe with x-ray
Consider scraping hacker news where each headline is made up of two TRs:
Geert-Jan
- 18,623
- 16
- 75
- 137
1
vote
1 answer
How do you dynamically pass in a cucumber feature file to a maven project Using the command prompt?
I've created maven project with selenium and cucumber. I'm trying to use jira X-ray in a continuous integration setup. Basically I take exported feature files and want to execute them on a command line using bamboo.
I think my main problem is I'm…
fractal
- 1,649
- 17
- 31
1
vote
1 answer
Scrape web with x-ray
I'm using x-ray to extract some data from a web site but when I get to the point to crawl to another page using the built-in functionality, it simply doesn't work.
UnitPrice is the parameter I want to extract but I get "undefined" all the time.
As…
Rober
- 726
- 8
- 27
1
vote
1 answer
How can get second item of array with x-ray
I want to get the second item in the game: ['.contenedor-numero'], array :
var site = 'http://www.laliga.es/en/';
var url = 'liga-bbva';
var address = site + url;
x(address, '#div_clasf_38_1_3 table tbody tr', [{
rank:…
Saleh
- 48
- 5
1
vote
1 answer
scrape hacker news via x-ray/node
How can i scrape hacker news (https://news.ycombinator.com/) via x-ray/nodejs?
I would like to get something like this out of it:
[
{title1, comment1},
{title2, comment2},
...
{"‘Minimal’ cell raises stakes in race to harness synthetic…
Rentrop
- 20,979
- 10
- 72
- 100
1
vote
1 answer
How do I specify 'all td, and all td img' in a CSS selector?
I am using the X-ray JS package to scrape a table from a page. Desired tags and attributes are specified using an array of CSS selectors in a JSON string.
"{[ 'th, td' ]}" correctly gets content from all th and td tags.
I additionally need the img…
Dizzley
- 487
- 6
- 16
1
vote
1 answer
Insert to db from Meteor server not working
I'm trying to run a simple script that will scrape some data using x-ray and insert it into my Events collection.
if (Meteor.isServer) {
var Xray = Meteor.npmRequire('x-ray');
var xray = new Xray({
version: "2.0.3"
…
S. Heutmaker
- 146
- 1
- 6
1
vote
2 answers
X Ray Scraper: Manipulate data before .write
I'm fiddling around with some scraping, and need to manipulate some of the data before writing it to my json file.
var Xray = require('x-ray');
var x = Xray();
x('http://myUrl.com', '#search_results div div a', [{
title:…
MartinDK81
- 343
- 2
- 8
- 16
1
vote
2 answers
Node.js Force to Wait for Function to Finish
I have a for-loop in a program I am running with Node.js. The function is x() from the xray package and I am using it to scrape and receive data from a webpage and then write that data to a file. This program is successful when used to scrape ~100…
Hannah Murphy
- 119
- 1
- 9
1
vote
1 answer
Rx.Observable.ForkJoin in RxJS and parallel async (using X-Ray)
I am trying to figure out how to run in parallel (in this case 10) async function based on a stream of parsing datas from a website using lapwinglabs/x-ray webscraper.
let pauser = new Rx.Subject()
let count = 0
let max = 10
// function that parse…
Koalabz
- 13
- 5
0
votes
1 answer
Multiple Jenkin execution results in replacing the result values in XRay issue
I am trying to integrate Junit, Jenkins with Xray such that post automation it has to update the Xray test cases. I have specified the dedicated Test Execution value so that after every execution, same issue needs to be updated with the Xray…
Gagan changappa
- 33
- 1
- 6
0
votes
0 answers
I want to get X-UI database information to run a Telegram Bot, I need properties like "total flow" &"expiration date" which are available in x-ui view
I want to get X-UI database information to run a Telegram Bot, I need properties like "total flow" and "expiration date" which are available in x-ui dashboard but I can not find this data neither in x-ui.db file or config.json file.
My x-ui project…
Farzad Emadi
- 11
- 1
0
votes
1 answer
Difference between Slope-Intercept and Hounsfield Scale in CT Dicom
This may come as a naive question. We know that CT computes the linear attenuation coefficient of each voxel and the corresponding Hounsfield number is calculated based on the formula: HU = (µmaterial–µwater)/(µwater) × 1000 (say eq.1). But in…
ozark_ddr
- 1
- 2