Stack Overflow
Stack Overflow

Questions tagged [xidel]

Xidel is a command line tool to download and extract data from HTML/XML pages as well as JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern templates. It can also edit or create new XML/HTML/JSON documents.

Xidel supports:

Extract expressions

  • CSS 3 Selectors: to extract simple elements
  • XPath 3.0: to extract values and calculate things with them
  • XQuery 3.0: to create new documents from the extracted values
  • JSONiq: to work with JSON apis
  • Templates: to extract several expressions in an easy way using an annotated version of the page for pattern-matching
  • XPath 2.0/XQuery 1.0: compatibility mode for the old XPath/XQuery version

Following

  • HTTP Codes: Redirections like 30x are automatically followed, while keeping things like cookies
  • Links: It can follow all links on a page as well as some extracted values
  • Forms: It can fill in arbitrary data and submit the form

Output formats

  • Adhoc: just prints the data in a human readable format
  • XML: encodes the data as XML
  • HTML: encodes the data as HTML
  • JSON: encodes the data as JSON
  • bash/cmd: exports the data as shell variables

Connections

  • HTTP / HTTPS, as well as local files and stdin

Systems

  • Windows (using wininet), Linux (using synapse+openssl), Mac (synapse)
81 questions
0
votes
0 answers

Xidel keeps adding XMLNS in the output

I am trying to extract html code from a div:
…
Adrian
  • 2,576
  • 9
  • 49
  • 97
0
votes
2 answers

How to get xpath of invalid html?

I am trying to extract xpath from html using xidel and recently encountered invalid html i use firefox to get the xpath, but firefox automatically adds missing tags so the xpath doesent match can i stop firefox, or can you suggest a way to deal with…
mata safute
  • 1
  • 1
0
votes
1 answer

Finding a specific link from a site

I'm trying to find a specific link from a web page using windows command line and tools. I think Xidel can do what I want to do. In the page, the link is used like this: file: 'http://link.link/index.txt' Note: there's only one line like this. Now…
Fahim F.
  • 5
  • 4
0
votes
2 answers

Pass external variable to xidel in bash loop script

I try to parse html page using XPath with xidel. The page have a table with multiple rows and columns I need to get values from each row from columns 2 and 5 (IP and port) and store them in csv-like file. Here is my script #!/bin/bash for (( i = 2;…
Exabyte
  • 53
  • 5
0
votes
1 answer

Using Xidel how to extract value from xml file

In the following XML file, I would like to extract the value 300. 300 I tried the following code but it doesn't seem to be working. xidel 1.xml -e…
Arun
  • 23
  • 1
  • 5
-2
votes
1 answer

"invalid type for union" of join()

$ xidel -s -e '//PubmedArticle/join((MedlineCitation/PMID, normalize-space(outer-xml())), x:cps(9))' 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&retmode=xml&id=22259814,26978846' 26978846
user1424739
  • 11,937
  • 17
  • 63
  • 152
1 2 3 4 5
6