write to database with tweetstream daemon

Question

I am trying to write all tweets that matches a keyword to my database. I have set up the following in tracker.rb:

require 'rubygems'
require 'tweetstream'

TweetStream::Daemon.new('Bill Gates','money','Twitter Tracker').track('ladygaga') do |status|
  Tweet.new(:content => status.text)
end

But nothing happens. What am I doing wrong here?

Thanks in advance

Update: I put everything in a .rake file called twitter.rake and start the demon with $ rake scrap:

task :scrap => :environment do
  desc "Run Twitter Scraper"
  TweetStream::Client.new('TWITTER_USER','TWITTER_PASS').track('ladygaga') do |status|
    Tweet.create(:user_id  => status.user.id, :user_screen_name => status.user.screen_name, :user_profile_image_url => status.user.profile_image_url, :status_text => status.text, :status_id => status.id)
    puts "[#{status.user.screen_name}] #{status.text}"
  end
end

Answer 1

How are you calling the Daemon?

You need to supply a command (start/stop..)

For example:

rails runner "TweetStream::Daemon.new('tracker').track('ladygaga') { |status| do_something }" start

This would start the job in the background

Answer 2

Your first approach was the best one, you need to run "deamon" from the command line, but since you want to user rails and the activerecord you need to bootstrap the rails environment in to the script.

You need to do something like this:

#!/usr/bin/env ruby
# encoding: utf-8

ENV["RAILS_ENV"] ||= "development"

root  = File.expand_path(File.join(File.dirname(__FILE__), '..'))
require File.join(root, "config", "environment")

require 'tweetstream'

p "Initializing daemon..."

TweetStream.configure do |config|
  config.consumer_key       = 'your-consumer_key'
  config.consumer_secret    = 'your-consumer_secret'
  config.oauth_token        = 'your-oauth_token'
  config.oauth_token_secret = 'your-oauth_token_secret'
  config.auth_method        = :oauth
end

terms = ['ladygaga']

daemon = TweetStream::Daemon.new('tracker',
  :log_output => true,
  :backtrace  => true,
)

daemon.on_inited do
  ActiveRecord::Base.connection.reconnect!
  p "Listening..."
end

daemon.on_error do |message|
  puts "on_error: #{message}"
end

daemon.on_reconnect do |timeout, retries|
  puts "on_reconnect: #{timeout}, #{retries}"
end

daemon.on_limit do |discarded_count|
  puts "on_limit: #{skip_count}"
end

daemon.track(terms) do |status|
  # put here your model.create code!
  # Tweet.create!( :uid => status.id, ... )
end

To run the script just type:

ruby scrip-name.rb run

Answer 3

I'm assuming this is part of a larger rails application. If so, issue 1 is that Tweet.new will not persist anything to the database if it is a standard activerecord object. Try Tweet.create Secondly I'm not sure if the script will necessarily know about the Tweet if its an activerecord without also pulling in the rails app, possibly by including the environment.rb file.

Something like:

ENV["RAILS_ENV"] ||= "production"

require File.dirname(__FILE__) + "/../../config/application"
Rails.application.require_environment!

If that doesn't work you could try just including active record theres a question and answer here that describes it:

How to use ActiveRecord in a ruby script outside Rails?