1

Context

Originally, my app was working properly on both my local and production server (Heroku). I then made a change that I prematurely pushed to Heroku; the change broke the version on production resulting in the showing of the "500" error. The app still worked on my local server.

To fix this, I ran the heroku rollback command to revert back to the earlier working version on production. The production version was now working. To make sure my local and production copies were synced with working versions, I thought I would simply run the heroku git:clone -a myapp command so that I could replace my local copy with the working production copy.

Problem

After replacing my local copy with the working copy from production, I then tried to push to Heroku (to verify everything was working). My local version still works properly; however, the production version simply shows the "505" error and running heroku rollback no longer rolls back to the earlier working version. How can I get the production version working?

Apologies for the circuitous explanation. Any help would be greatly appreciated!

Gemfile:

source 'https://rubygems.org'

gem 'rails', '3.2.13'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

group :development do
  gem 'sqlite3', '1.3.5'
end


# Gems used only for assets and not required
# in production environments by default.
group :assets do
  gem 'sass-rails',   '~> 3.2.3'
  gem 'coffee-rails', '~> 3.2.1'

  # See https://github.com/sstephenson/execjs#readme for more supported runtimes
  # gem 'therubyracer', :platforms => :ruby

  gem 'uglifier', '>= 1.0.3'
  # gem 'twitter-bootstrap-rails'
  gem 'libv8', '~> 3.11.8'
  gem 'less-rails'
  gem 'therubyracer'
end

gem 'jquery-rails'
gem 'twitter-bootstrap-rails'

# To use ActiveModel has_secure_password
# gem 'bcrypt-ruby', '~> 3.0.0'

# To use Jbuilder templates for JSON
# gem 'jbuilder'

# Use unicorn as the app server
# gem 'unicorn'

# Deploy with Capistrano
# gem 'capistrano'

# To use debugger
# gem 'debugger'
group :production do
  gem 'pg', '0.12.2'
end

Output of running heroku logs command:

2013-06-05T21:54:39.324301+00:00 heroku[api]: Enable Logplex by ...@...com
2013-06-05T21:54:39.365512+00:00 heroku[api]: Release v2 created by ...@...com
2013-06-05T22:05:00.052911+00:00 heroku[slugc]: Slug compilation started
2013-06-05T22:07:13.861177+00:00 heroku[api]: Scale to web=1 by ...@...com
2013-06-05T22:07:14.437288+00:00 heroku[api]: Attach HEROKU_POSTGRESQL_BROWN resource by ...@...com
2013-06-05T22:07:14.466434+00:00 heroku[api]: Release v3 created by ...@...com
2013-06-05T22:07:14.531348+00:00 heroku[api]: Add DATABASE_URL config by ...@...com
2013-06-05T22:07:14.562176+00:00 heroku[api]: Release v4 created by ...@...com
2013-06-05T22:07:14.615023+00:00 heroku[api]: Add  config by ...@...com
2013-06-05T22:07:14.647163+00:00 heroku[api]: Release v5 created by ...@...com
2013-06-05T22:07:14.690706+00:00 heroku[api]: Deploy 0781b19 by ...@...com
2013-06-05T22:07:14.720746+00:00 heroku[api]: Release v6 created by ...@...com
2013-06-05T22:07:14.758968+00:00 heroku[api]: Deploy 0781b19 by ...@...com
2013-06-05T22:07:15.391452+00:00 heroku[slugc]: Slug compilation finished
2013-06-05T22:07:17.270346+00:00 heroku[web.1]: Starting process with command `bundle exec rails server -p 24519`
2013-06-05T22:07:21.081962+00:00 app[web.1]: DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /app/config/environment.rb:5)
2013-06-05T22:07:21.081844+00:00 app[web.1]: DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /app/config/environment.rb:5)
2013-06-05T22:07:24.020299+00:00 app[web.1]: => Booting WEBrick
2013-06-05T22:07:24.020299+00:00 app[web.1]: => Ctrl-C to shutdown server
2013-06-05T22:07:24.020299+00:00 app[web.1]: => Call with -d to detach
2013-06-05T22:07:24.020299+00:00 app[web.1]: => Rails 3.2.13 application starting in production on http://0.0.0.0:24519
2013-06-05T22:07:24.020299+00:00 app[web.1]: Connecting to database specified by DATABASE_URL
2013-06-05T22:07:24.195433+00:00 app[web.1]: [2013-06-05 22:07:24] INFO  ruby 1.9.2 (2011-07-09) [x86_64-linux]
2013-06-05T22:07:24.197238+00:00 app[web.1]: [2013-06-05 22:07:24] INFO  WEBrick::HTTPServer#start: pid=2 port=24519
2013-06-05T22:07:24.195268+00:00 app[web.1]: [2013-06-05 22:07:24] INFO  WEBrick 1.3.1
2013-06-05T22:07:24.563524+00:00 heroku[web.1]: State changed from starting to up
2013-06-05T22:20:29.360706+00:00 app[web.1]: Started GET "/" for 108.94.165.104 at 2013-06-05 22:20:29 +0000
2013-06-05T22:20:29.573466+00:00 app[web.1]: Processing by StudentsController#index as HTML
2013-06-05T22:20:29.721672+00:00 app[web.1]: Completed 500 Internal Server Error in 148ms
2013-06-05T22:20:29.729040+00:00 app[web.1]:                      pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
2013-06-05T22:20:29.729040+00:00 app[web.1]:                                         ^
2013-06-05T22:20:29.729040+00:00 app[web.1]: :             SELECT a.attname, format_type(a.atttypid, a.atttypmod),
2013-06-05T22:20:29.729040+00:00 app[web.1]: 
2013-06-05T22:20:29.729040+00:00 app[web.1]:               FROM pg_attribute a LEFT JOIN pg_attrdef d
2013-06-05T22:20:29.729040+00:00 app[web.1]: ActiveRecord::StatementInvalid (PGError: ERROR:  relation "students" does not exist
2013-06-05T22:20:29.729040+00:00 app[web.1]:                 ON a.attrelid = d.adrelid AND a.attnum = d.adnum
2013-06-05T22:20:29.729040+00:00 app[web.1]: LINE 5:              WHERE a.attrelid = '"students"'::regclass
2013-06-05T22:20:29.729040+00:00 app[web.1]:              WHERE a.attrelid = '"students"'::regclass
2013-06-05T22:20:29.729327+00:00 app[web.1]:              ORDER BY a.attnum
2013-06-05T22:20:29.729327+00:00 app[web.1]: 
2013-06-05T22:20:29.729327+00:00 app[web.1]: ):
2013-06-05T22:20:29.729327+00:00 app[web.1]:   app/controllers/students_controller.rb:5:in `index'
2013-06-05T22:20:29.729040+00:00 app[web.1]:                AND a.attnum > 0 AND NOT a.attisdropped
2013-06-05T22:20:29.729327+00:00 app[web.1]: 
2013-06-05T22:20:29.732475+00:00 heroku[router]: at=info method=GET path=/ host=secret-brushlands-9122.herokuapp.com fwd="108.94.165.104" dyno=web.1 connect=7ms service=385ms status=500 bytes=643
2013-06-05T22:20:29.907973+00:00 heroku[router]: at=info method=GET path=/favicon.ico host=secret-brushlands-9122.herokuapp.com fwd="108.94.165.104" dyno=web.1 connect=1ms service=11ms status=200 bytes=0
2013-06-05T22:29:58.144871+00:00 heroku[api]: Deploy 0403476 by ...@...com
2013-06-05T22:29:58.167804+00:00 heroku[api]: Release v7 created by ...@...com
2013-06-05T22:29:58.206118+00:00 heroku[api]: Deploy 0403476 by ...@...com
2013-06-05T22:29:58.376314+00:00 heroku[web.1]: State changed from up to starting
2013-06-05T22:30:01.539107+00:00 heroku[web.1]: Starting process with command `bundle exec rails server -p 52191`
2013-06-05T22:30:01.701845+00:00 heroku[web.1]: Stopping all processes with SIGTERM
2013-06-05T22:30:02.512228+00:00 app[web.1]: [2013-06-05 22:30:02] ERROR SignalException: SIGTERM
2013-06-05T22:30:02.512228+00:00 app[web.1]:    /usr/local/lib/ruby/1.9.1/webrick/server.rb:90:in `select'
2013-06-05T22:30:12.438954+00:00 heroku[web.1]: Error R12 (Exit timeout) -> At least one process failed to exit within 10 seconds of SIGTERM
2013-06-05T22:30:12.439155+00:00 heroku[web.1]: Stopping remaining processes with SIGKILL
2013-06-05T22:30:14.361060+00:00 heroku[web.1]: Process exited with status 137
2013-06-05T22:30:15.586713+00:00 app[web.1]: DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /app/config/environment.rb:5)
2013-06-05T22:30:15.586566+00:00 app[web.1]: DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /app/config/environment.rb:5)
2013-06-05T22:30:35.092923+00:00 app[web.1]: => Call with -d to detach
2013-06-05T22:30:35.092923+00:00 app[web.1]: => Ctrl-C to shutdown server
2013-06-05T22:30:35.092923+00:00 app[web.1]: Connecting to database specified by DATABASE_URL
2013-06-05T22:30:35.092923+00:00 app[web.1]: => Booting WEBrick
2013-06-05T22:30:35.092923+00:00 app[web.1]: => Rails 3.2.13 application starting in production on http://0.0.0.0:52191
2013-06-05T22:30:35.594816+00:00 app[web.1]: [2013-06-05 22:30:35] INFO  WEBrick 1.3.1
2013-06-05T22:30:35.594816+00:00 app[web.1]: [2013-06-05 22:30:35] INFO  ruby 1.9.2 (2011-07-09) [x86_64-linux]
2013-06-05T22:30:35.595558+00:00 app[web.1]: [2013-06-05 22:30:35] INFO  WEBrick::HTTPServer#start: pid=2 port=52191
2013-06-05T22:30:35.990236+00:00 heroku[web.1]: State changed from starting to up
2013-06-05T22:30:38.042484+00:00 app[web.1]: Started GET "/" for 108.94.165.104 at 2013-06-05 22:30:38 +0000
2013-06-05T22:30:38.423594+00:00 app[web.1]: Processing by StudentsController#index as HTML
2013-06-05T22:30:39.528941+00:00 heroku[router]: at=info method=GET path=/ host=secret-brushlands-9122.herokuapp.com fwd="108.94.165.104" dyno=web.1 connect=2ms service=1523ms status=500 bytes=643
2013-06-05T22:30:39.706864+00:00 heroku[router]: at=info method=GET path=/favicon.ico host=secret-brushlands-9122.herokuapp.com fwd="108.94.165.104" dyno=web.1 connect=2ms service=24ms status=200 bytes=0
2013-06-05T22:30:39.512419+00:00 app[web.1]: Completed 500 Internal Server Error in 1088ms
2013-06-05T22:30:39.514250+00:00 app[web.1]: 
2013-06-05T22:30:39.514250+00:00 app[web.1]:                 ON a.attrelid = d.adrelid AND a.attnum = d.adnum
2013-06-05T22:30:39.514430+00:00 app[web.1]:   app/controllers/students_controller.rb:5:in `index'
2013-06-05T22:30:39.514250+00:00 app[web.1]:                AND a.attnum > 0 AND NOT a.attisdropped
2013-06-05T22:30:39.514430+00:00 app[web.1]:              ORDER BY a.attnum
2013-06-05T22:30:39.514430+00:00 app[web.1]: ):
2013-06-05T22:30:39.514250+00:00 app[web.1]: LINE 5:              WHERE a.attrelid = '"students"'::regclass
2013-06-05T22:30:39.514250+00:00 app[web.1]:                                         ^
2013-06-05T22:30:39.514250+00:00 app[web.1]: ActiveRecord::StatementInvalid (PGError: ERROR:  relation "students" does not exist
2013-06-05T22:30:39.514250+00:00 app[web.1]: :             SELECT a.attname, format_type(a.atttypid, a.atttypmod),
2013-06-05T22:30:39.514250+00:00 app[web.1]:                      pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
2013-06-05T22:30:39.514250+00:00 app[web.1]:               FROM pg_attribute a LEFT JOIN pg_attrdef d
2013-06-05T22:30:39.514250+00:00 app[web.1]:              WHERE a.attrelid = '"students"'::regclass
2013-06-05T22:30:39.514430+00:00 app[web.1]: 
2013-06-05T22:30:39.514430+00:00 app[web.1]: 

UPDATE:

I was curious if creating a new Heroku remote would solve the problem so I deleted http://nameless-brook-8163.herokuapp.com/ and set the new remote to http://secret-brushlands-9122.herokuapp.com/. Unfortunately, the problem persists.

I ran 'heroku run rake db:migrate' per a recommendation below and got the following output:

heroku run rake db:migrate
Running `rake db:migrate` attached to terminal... up, run.6596
DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /app/Rakefile:7)
DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /app/Rakefile:7)
rake aborted!
Invalid DATABASE_URL
(erb):9:in `rescue in <main>'
(erb):6:in `<main>'
/app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/application/configuration.rb:115:in `database_configuration'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/railtie.rb:78:in `block (2 levels) in <class:Railtie>'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/lazy_load_hooks.rb:36:in `instance_eval'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/lazy_load_hooks.rb:36:in `execute_hook'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/lazy_load_hooks.rb:43:in `block in run_load_hooks'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/lazy_load_hooks.rb:42:in `each'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/lazy_load_hooks.rb:42:in `run_load_hooks'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/base.rb:720:in `<top (required)>'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/railties/databases.rake:25:in `block (2 levels) in <top (required)>'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `call'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `block in execute'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `each'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `execute'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:184:in `block in invoke_with_call_chain'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:177:in `invoke_with_call_chain'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:205:in `block in invoke_prerequisites'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:203:in `each'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:203:in `invoke_prerequisites'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:183:in `block in invoke_with_call_chain'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:177:in `invoke_with_call_chain'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:170:in `invoke'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:143:in `invoke_task'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block (2 levels) in top_level'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `each'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block in top_level'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:110:in `run_with_threads'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:95:in `top_level'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:73:in `block in run'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:160:in `standard_exception_handling'
/app/vendor/bundle/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:70:in `run'
Tasks: TOP => db:migrate => db:load_config
(See full trace by running task with --trace)
Thai Nguyen
  • 55
  • 1
  • 7
  • it looks like the asset-pipeline did not work on deployment. just trigger a new deployment on heroku by pushing a new commit. – phoet Jun 05 '13 at 22:25
  • Hi @phoet, as you suggested, I pushed a new commit to Heroku. The 500 error still arises. I updated the output of running 'heroku logs'. This caught my attention: '2013-06-05T22:20:29.729040+00:00 app[web.1]: ActiveRecord::StatementInvalid (PGError: ERROR: relation "students" does not exist'. Do you have an idea of how I should interpret this? Thanks! – Thai Nguyen Jun 05 '13 at 22:40
  • heroku run db:migrate – phoet Jun 05 '13 at 22:47
  • @phoet, I ran 'heroku run db:migrate'; the output was: Running `db:version` attached to terminal... up, run.6716 bash: db:version: command not found – Thai Nguyen Jun 05 '13 at 23:02
  • It's definitely a migration error. Are you running pg locally too? – XanderStrike Jun 05 '13 at 23:08
  • Hi @XanderStrike, I'm running SQLite locally. – Thai Nguyen Jun 05 '13 at 23:28
  • How about "heroku run rake db:migrate" – Josh Jun 06 '13 at 01:54
  • See the help for `heroku rollback` - you can rollback to a different release, not just the "previous" one. That should get your production going again. I suspect when you did check your local git repo and roll that back to a previous version too. Notice that when you do `heroku releases` it shows a git commit for each release. That will allow you to also roll your local back to the appropriate place. – Jon Mountjoy Jun 06 '13 at 11:35
  • Hi @Josh, the command executed the db migration but, unfortunately, the "500" error still shows :-(. – Thai Nguyen Jun 06 '13 at 17:29
  • try `heroku releases` to see what commit is live. Also, try `heroku restart` after you run the migration. – Josh Jun 06 '13 at 20:31
  • Check your heroku logs again, this 500 error may have a different cause. If you can, also use postgres in development. This may save you heaps of time when doing delicate database queries that work on sqlite but not on postgres. – fivedigit Jun 08 '13 at 05:27
  • Hi @Josh, I ran 'heroku run rake db:migrate' and got the output above (please see the 'update'). Do you think the deprecated plug-ins is the cause? – Thai Nguyen Jun 10 '13 at 22:29
  • No, the deprication warnings are from Heroku and are NOT causing your problem. You don't have a DB set up for that app, or it's not configured correctly: http://stackoverflow.com/questions/8659905/heroku-on-rails-invalid-database-url – Josh Jun 11 '13 at 06:47
  • @Josh, thanks for the reference. I tried to reset the DB URL with the Postgres URL: heroku config:set DATABASE_URL=postgres://your-db-path. The only change was that before I was seeing the following error on the live site: "An error occurred in the application and your page could not be served. Please try again in a few moments. If you are the application owner, check your logs for details." and now I just see a 500 error. – Thai Nguyen Jun 11 '13 at 17:33

1 Answers1

0

The problem appears to be an open comment (*/) in one of my CSS files. This came to my attention when I attempted to compile the assets with the command:

RAILS_ENV=production bundle exec rake assets:precompile

and got the error...

rake aborted!
Invalid CSS after "*/": expected identifier, was "/* Sections"

I deleted the extraneous "*/", forced a push to Heroku (git push -f heroku master), deleted index.html and voila the right page was served.

Thank you to all for the help and recommendations!

Thai Nguyen
  • 55
  • 1
  • 7
  • Thanks! This helped me tremendously. I also had an issue with an incorrect formatted CSS files in a vendor/assets file. Unfortunately, it was in a minified file, so tracking down the missing semicolon was a little annoying. – Ryan Jul 09 '15 at 22:49