Jobs update with Dashing and Ruby

Question

I use Dashing for monitor trends and website statistics. I create a jobs to check GooglesNews trends and Twitter trends .

The data is displayed well, however, they appear at first load and does put more update then. There is the code for twitter_trends.rb :

require 'nokogiri'
require 'open-uri'

url = 'http://trends24.in/france/~cloud'

data = Nokogiri::HTML(open(url))
list = data.xpath('//ol/li')

tags = list.collect do |tag|
  tag.xpath('a').text
end

tags = tags.take(10)
tag_counts = Hash.new({value: 0})

SCHEDULER.every '10s' do
  tag = tags.sample
  tag_counts[tag] = {label: tag}

  send_event('twitter_trends', {items: tag_counts.values})
end

I think I used bad "rufus-scheduler" to schedule my job jobs https://gist.github.com/pushmatrix/3978821#file-sample_job-rb

How to make the data will update correctly on a regular basis ?

Why is rufus-scheduler bad? Romain, pourquoi qualifies-tu rufus-scheduler de mauvais? — jmettraux, Feb 11 '16 at 21:16
I meant that I do not know to use it. I do not know how my data are updated regularly. rufus-scheduler is good :) — Romain Schneider, Feb 12 '16 at 08:33

score 2 · Answer 1 · answered Feb 12 '16 at 19:47

2

Your scheduler looks fine, but it looks like you're making one call to the website:

data = Nokogiri::HTML(open(url))

But never calling it again. Is your intent to only check that site once along with the initial processing of it?

I assume you'd really want to wrap more of your logic into the scheduler loop - only things in there will be rerun when the schedule job hits.

answered Feb 12 '16 at 19:47

Michael Knowles

66
1
4

My problem is here. I don't want to call will be done once just one time. If I move `SCHEDULER.every ' 10s ' do` over `data = HTML :: Nokogiri (open ( url) )` the script no longer works. – Romain Schneider Feb 12 '16 at 21:06
1

What error are you getting in in the log file? It may be worth adding a "puts tag.to_s" or such in there, which will also show up in the logs so you can see it more clearly. – Michael Knowles Feb 12 '16 at 21:08

score 0 · Answer 2 · edited Apr 08 '16 at 15:04

When you covered everything in a scheduler, you are only taking one sample every 10 seconds (http://ruby-doc.org/core-2.2.0/Array.html#method-i-sample) then adding it to tag_counts. This is clearing the tag each time. Thing to remember about schedulers is it's basically a clean slate every time it runs. I'd recommend looping through tags and adding them to tag_counts that way instead of sampling. sampling is kind of unnecessary seeing as you are reducing it to 10 each time you run the scheduler.

score -1 · Answer 3 · answered Feb 13 '16 at 16:43

If I move the SCHEDULER like this (after url on top), it works but that only one item appears randomly every 10 seconds.

require 'nokogiri'
require 'open-uri'

url = 'http://trends24.in/france/~cloud'

SCHEDULER.every '10s' do

data = Nokogiri::HTML(open(url))
list = data.xpath('//ol/li')

tags = list.collect do |tag|
  tag.xpath('a').text
end

tags = tags.take(10)
tag_counts = Hash.new({value: 0})

  tag = tags.sample
  tag_counts[tag] = {label: tag}

  send_event('twitter_trends', {items: tag_counts.values})
end

How to display a list of 10 items, which is updated regularly ?

Jobs update with Dashing and Ruby

3 Answers3