Mongodb not inserting Ruby time.new consistantly on Heroku

Question

Built a small app to grab Tweets from political candidates for the upcoming election. Using Ruby, Twitterstream, Mongodb and Heroku.

The time is being inserted into the database inconsistantly. Sometimes it works, sometimes it doesn't. Is this my code, Heroku or Mongodb (Mongohq). I have a support question in.

Working

{
  _id: ObjectId("52556b5bd2d9530002000002"),
  time: ISODate("2013-10-09T14:42:35.044Z"),
  user: "Blondetigressnc",
  userid: 1342776674,
  tweet: "RT @GovBrewer: Mr. President @BarackObama, reopen America’s National Parks or let the states do it. #GrandCanyon #Lead http://t.co/kkPKt9B7…",
  statusid: "387951226866110464"
}

Not working

{
  _id: ObjectId("52556c2454d4ad0002000016"),
  user: "PeterMcC66",
  userid: 1729065984,
  tweet: "@GovBrewer @Blondetigressnc @BarackObama Time to impeach surely?",
  statusid: "387952072223506432"
}

Seems random. See anything wrong or stupid in my code?

require 'rubygems'
require 'tweetstream'
require 'mongo'

# user ids 
users = 'list of Twitter user ids here'

# connect to stream
TweetStream.configure do |config|
  config.consumer_key       = ENV['T_KEY']
  config.consumer_secret    = ENV['T_SECRET']
  config.oauth_token        = ENV['T_TOKEN']
  config.oauth_token_secret = ENV['T_TOKEN_SECRET']
  config.auth_method        = :oauth
end

# connection to database
  if ENV['MONGOHQ_URL']
    uri = URI.parse(ENV['MONGOHQ_URL'])
    conn = Mongo::Connection.from_uri(ENV['MONGOHQ_URL'])
    DB = conn.db(uri.path.gsub(/^\//, ''))
  else
    DB = Mongo::Connection.new.db("tweetsDB")
  end

# creation of collections
tweets =  DB.create_collection("tweets")
deleted = DB.create_collection("deleted-tweets")


@client = TweetStream::Client.new

@client.on_delete do | status_id, user_id |
  puts "#{status_id}"
  timenow = Time.new
  id = status_id.to_s
  deleted.insert({ :time => timenow, :user_id => user_id, :statusid => id })
end

@client.follow(users) do |status|
  puts "[#{status.user.screen_name}] #{status.text}"
  timenow = Time.new
  id = status.id
  tweets.insert({ :time => timenow, :user => status.user.screen_name, :userid => status.user.id, :tweet => status.text, :statusid => id.to_s })
end

score 1 · Answer 1 · answered Dec 04 '13 at 06:27

The issue is that you need to use a UTC time, not your local timezone. This is not a MongoDB or a Ruby driver issue, its a constraint of the BSON spec and the ISODate BSON type.

http://docs.mongodb.org/manual/reference/bson-types/#date
http://bsonspec.org/#/specification

Also, just a good practice though.

General word of advice: Use UTC on the back-end of whatever you're building always anyway, regardless of what datastore you're using (not a MongoDB specific thing). This is especially true if this data is something you want to query on directly.

If you need to convert to a local timezone, its best to handle that when you display or output the data rather than trying to manage that elsewhere. Some of the most fantastic bugs I've ever seen were related to inconsistent handling of timezones in the persistence layer of the application.

Keep those times consistent on the back-and, deal with local timezone conversion when in your application and life will be much easier for you.

Here is an examples of how to work with times in MongoDB using Ruby:

require 'time' # required for ISO-8601
require 'mongo'

include Mongo
client = MongoClient.new
coll   = client['example_database']['example_collection']

coll.insert({ 'updated_at' => Time.now.utc })

doc = coll.find_one()

doc['updated_it'].is_a?(Time)                     #=> true
doc['updated_at'].to_s                            #=> "2013-10-07 22:43:52 UTC"
doc['updated_at'].iso8601                         #=> "2013-10-07T22:43:52Z"
doc['updated_at'].strftime("updated at %m/%d/%Y") #=> "updated at 10/07/2013"

I keep a gist of this available here:
https://gist.github.com/brandonblack/6876374

Mongodb not inserting Ruby time.new consistantly on Heroku

1 Answers1