0

I am using a rakefile to fetch information from one website and saving it to my database.

Using the TMDB-Gem, this code @movie = TmdbMovie.browse(:order_by => "release", :order => "asc", :page => 1, :per_page => 2, :language => "en", :expand_results => true) browses the oldest movies (:order_by => "release") and saves them to my database, but as i'll be running this rake quite frequently, the returned and saved movies will be the same.

Every movie has a tmdb_id and every id is unique

How can i make the rakefile check that the returned movie's tmdb_id is unique, and if there is already a movie with that id, skip and save the next movie.

I tried it in my Movies model validates_uniqueness_of :tmdb_id but it gives an error when running the rake command and it doesn't save movies.

Basically, how can I, through the rakefile, validate the uniqueness of the tmdb_id

This is my rake file

namespace :db do 

task :pull_tmdb_data => :environment do 

Tmdb.api_key = "API KEY" 



Tmdb.default_language = "en" 

  @movie = TmdbMovie.browse(:order_by => "release", :order => "asc", :page => 1, :per_page => 2, :language => "en", :expand_results => true)



@movie.each do |movie| 

Movie.create(title: movie.name, description: movie.overview, release_date: movie.released, tmdb_id: movie.id) 

end 



end 

end
PMP
  • 231
  • 6
  • 25

1 Answers1

1

You're right to use validates_uniqueness_of :tmdb_id in your Movie model. You didn't show how you were saving the movies to your database, but .save doesn't cause an exception when validations fail, whereas .save! does. The key would be to use a save method that doesn't raise an error when validations fail.

Edit - Now that I understand what you're actually trying to do, you should be able to do something like:

per_page = 100
number_of_movies = Movie.count
page = number_of_movies/per_page+1
@movies = TmdbMovie.browse(:order_by => "release", :order => "asc", :page => page, :per_page => per_page, :language => "en", :expand_results => true) browses the oldest movies (:order_by => "release")

So if you've already pulled 435 movies, it'll only return movies 400-500 in the next call .. I did it this way because I wasn't sure if there was an offset option, but if there is you could just offset the query by Movie.count, which would be better.

Mike Campbell
  • 7,921
  • 2
  • 38
  • 51
  • Added my rake file to the question. Using `validates_uniqueness_of :tmdb_id` causes the rake command to NOT save the returned movies, whereas if i remove `validates_uniqueness_of :tmdb_id` it saves it – PMP Jul 28 '13 at 14:11
  • `create` shouldn't raise an exception, can you please show us the error you get? – Mike Campbell Jul 28 '13 at 14:40
  • It doesnt return an error. What happens is that, if i dont have the validates... in the model, then push to heroku and run the rake command, it saves the returned movies to my database, like it should do. BUT, if i add validates_uni... to my model, then push to heroku and run the rake command, it DOESNT save the returned movies to my database – PMP Jul 28 '13 at 14:44
  • So you lied in your question? _"I tried it in my Movies model validates_uniqueness_of :tmdb_id but it gives an error when running the rake command and it doesn't save movies"_? What do you want to happen and what isn't happening currently that you want to happen? – Mike Campbell Jul 28 '13 at 14:47
  • I wouldn't call that lying, just a mistake. – PMP Jul 28 '13 at 14:50
  • Ok, mistake aside, what are you trying to achieve, what's the problem? I still don't understand the question. You say in your question: _"How can i make the rakefile check that the returned movie's tmdb_id is unique, and if there is already a movie with that id, skip and save the next movie."_, surely this is what happens with the validation in the model? – Mike Campbell Jul 28 '13 at 14:51
  • I run the rake command `rake db:pull_tmdb_data`, what this does is it browses **the 2 oldest movies** then saves them. If i run this again, it saves those same 2 movies, what i want it to do is save the 2 other oldest movies and not the ones already saved. If i put validates_uniquess.. in my model, in doesnt save any returned movie. – PMP Jul 28 '13 at 14:58
  • Oh, that has got nothing to do with validations, that's just because you're pulling the same TMDB data, you'll need to adjust your query to pull different records... `:page => 2` perhaps? The whole thing seems like a bad idea anyway. Why are you only pulling two records each time? – Mike Campbell Jul 28 '13 at 15:02
  • Thats just for testing purposes, when i launch it will obviously run to return more movies. But say i collect 100 movies, run that command, then save 100 movies. – PMP Jul 28 '13 at 15:03
  • What if a new movie arrives, then i run it again and i'll have 201 movies. The old 100 movies + the new 101 returned movies – PMP Jul 28 '13 at 15:04