Xidel: How to process multiple result from one node?

Question

Say, we have a code:

xidel -s https://www.example.com -e '(//span[@class="number"])'

and the output are:

111111
222222
333333

can I do this one below?

for ((n=1;n<=3;n++))
do
   a=$(xidel -s https://www.example.com -e '(//span[@class="number"]) [$n]')
   b=$a+1
   echo $b
done

I expect it to print out 3 edited numbers like this:

111112
222223
333334

it might be a little absurb to download the webpage 3 times, but the reason here is to process each value of the output one by one, using ForLoop.

You don't want a solution that downloads a whole webpage 8 times over just to print 8 different nodes. You want a solution that downloads the webpage *once* and selects 8 different nodes. The first thing you should do is get rid of the `for` loop. — Tomalak, Jun 16 '20 at 05:29
yeah, thats what i thought too, but how to get around with the [number] thing, after the node? — CuriousNewbie, Jun 16 '20 at 05:34
like this? `(//span[@class="number"]) [position() <= 8]`? the reason for me to use for loop is to process each output value, not just printing it. — CuriousNewbie, Jun 16 '20 at 05:40
*That's* your real question. *"How to process multiple result nodes from xidel?"* and even though I don't know xidel, I am quite sure that this is covered in the documentation. — Tomalak, Jun 16 '20 at 05:42
Also check out the documentation. This is such a common requirement that it's virtually impossible that this is not explained or shown. — Tomalak, Jun 16 '20 at 05:48
@Tomalak done that, thanks for reply btw. I already have my own answer in the post, ultimately I use forLoop to solve. — CuriousNewbie, Jun 16 '20 at 07:19

CuriousNewbie · Answer 1 · 2020-06-16T23:52:09.747

Example code:

$ aa=$(xidel -se '//span[@class="random"]' 'https://www.example.com')
$ echo $aa

Lets say that the result of xidel is this below:

a abc
a sdf
a wef
a vda
a gdr

and...lets say we want to cut all the a from each word of this list in this case, not limiting to just excluding the a.

We can use For Loop formula like this:

#"a " is the one we want to remove, so make variable for this prefix
a="a "

for ((n=-1;n>=-5;n--))
do
 #process the extraction by selecting which line first
   bb=$(echo "$aa" | head $n | tail -1)
 #then remove the prefix after that
   bb=${aa/#$a}
   echo $bb

done

This will print:

abc
sdf
wef
vda
gdr

Bonus

#"a " is the one we want to remove, so make variable for this prefix
a="a "

for ((n=-1;n>=-5;n--))
do
 #process the extraction by selecting which line first
   bb=$(echo "$aa" | head $n | tail -1)
 #then remove the prefix after that
   bb=${aa/#$a}
 #echo everything except 2nd line
 if [ $n != -2 ] ; then
 echo $bb
 fi

done

This will print:

abc
wef
vda
gdr

Any other input is welcome

score 1 · Accepted Answer · answered Jun 16 '20 at 22:31

1

xidel fully supports XPath/XQuery 3.0 (support for XPath/XQuery 3.1 is in development), so you can use all the features and filters it has to offer.
I can recommend the following websites:

Without a "Minimal, Reproducible Example" I'll just put your above mentioned output in a sequence and show you some examples.

xidel -se 'let $a:=(111111,222222,333333) return $a ! (. + 1)'
#or
xidel -se 'for $x in (111111,222222,333333) return $x + 1'
111112
222223
333334

xidel -se 'let $a:=("a abc","a sdf","a wef","a vda","a gdr") return $a ! substring-after(.,"a ")'
#or
xidel -se 'let $a:=("a abc","a sdf","a wef","a vda","a gdr") return $a ! replace(.,"a ","")'
#or
xidel -se 'for $x in ("a abc","a sdf","a wef","a vda","a gdr") return substring-after($x,"a ")'
#or
xidel -se 'for $x in ("a abc","a sdf","a wef","a vda","a gdr") return replace($x,"a ","")'
abc
sdf
wef
vda
gdr

answered Jun 16 '20 at 22:31

Reino

3,203
1
13
21

thank you @Reino, for your brilliant answer. although, it seems that xidel `let` does not support `if else` statrment, it fully answered my post question. – CuriousNewbie Jun 16 '20 at 23:36
@AhiungLim Please elaborate. If-statements shouldn't be a problem in a let-expression. – Reino Jun 17 '20 at 21:43
lets use this code below here: `xidel -se '//div[@class="product-box"]//span[@class="n-heading"]' 'https://www.anekalogam.co.id/id'` it should print 16 prices in Rupiah, So what we should do is first remove the "Rp " and the ".", then for line 3 & 4, the value divide by 2, for line 5 & 6, the value divide by 3, and so on with these sequence (1,2,3,5,10,25,50,100), since line 1 & 2 is already in value of 1 so it does not need to be divided. After all that, we need to echo it side by side like 1 & 2, 3 & 4, etc. like this for example: `905000 - 825000 853000 - 825000 ...` – CuriousNewbie Jun 18 '20 at 01:59
1

@AhiungLim I see where you're going. This honestly is worth a new question if you ask me. But anyway, is this, https://gist.github.com/Reino17/2f166849a0ebe8c64a36613723ebac64, what you're looking for? Btw, please specify input before an (extraction-)query. – Reino Jun 18 '20 at 22:36
omg @Reino you are trully expert in xidel, arent you? I humbly gratitude for your excellent answer. – CuriousNewbie Jun 19 '20 at 00:36
btw, @Reino, can you explain to me the expressions you used in the code? or I have to make new question about it? – CuriousNewbie Jun 19 '20 at 08:18
@AhiungLim You're welcome. The real expert though is `xidel`'s author, Benito. I've learned a lot from him. I think it's better if you'd post a new question, yes. Alternatively you could also [subscribe to the mailinglist, or visit the forums](http://videlibri.sourceforge.net/xidel.html#contact). As the forums aren't particularly active, I'd recommend the mailinglist. – Reino Jun 19 '20 at 10:43

Xidel: How to process multiple result from one node?

2 Answers2