Postgres error on Heroku with Resque

Question

I don't know enough Postgres to understand the message.

PG::Error: SSL error: decryption failed or bad record mac : SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc, a.attnotnull FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum WHERE a.attrelid = '"users"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum

And here is the callstack got from the resque backend

/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1139:in `async_exec'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1139:in `exec_no_cache'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:663:in `block in exec_query'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:280:in `block in log'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.2/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:275:in `log'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:662:in `exec_query'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1264:in `column_definitions'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:858:in `columns'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/schema_cache.rb:12:in `block in initialize'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `yield'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `default'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `columns'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:237:in `columns_hash'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/delegation.rb:7:in `columns_hash'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:330:in `find_one'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:311:in `find_with_ids'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:107:in `find'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/querying.rb:5:in `find'
/app/app/workers/compute_worker.rb:5:in `perform'

Explanation: My worker is only doing some writing operations into the databases. Nothing complicated. When I execute the worker's task for the first time, it works, and the second time it fails with the above message. If I restart the worker, it will work only the first time. I already run

heroku pg:restart
heroku db:restart
heroku db:migrate

... without success.

On my local computer(using mysql) it works perfectly. Any idea ?

Take a look at your column types in MySQL. Maybe you have not specified one column. I have had a similar error(long time ago) — Rails beginner, Apr 01 '12 at 01:13
That query is what ActiveRecord uses to figure out the structure of a PostgreSQL table. The error has [appeared elsewhere on SO](http://stackoverflow.com/questions/9108680/rails-postgresql-ssl-decryption-failure). — mu is too short, Apr 01 '12 at 01:31
Thanks for your comments. In fact I don't understand why it works the first time and fails as of the second time. I'm not using taps. Why should I use it ? — Arkan, Apr 01 '12 at 12:41

score 14 · Accepted Answer · answered Apr 01 '12 at 13:57

14

Thanks to the comments of "Rails beginner" and "mu is too short" I found the solution.

I've added this to an initializer and it did the trick!

Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }

answered Apr 01 '12 at 13:57

Arkan

6,196
3
38
54

It actually required before_fork for my issue, so my solution ended up like this. – Paul Pettengill Apr 07 '12 at 10:56
Arkan - this solution saved my life. Well, at least my weekend. – rockusbacchus Dec 30 '16 at 22:48

score 7 · Answer 2 · answered Apr 07 '12 at 10:58

7

I had the same issue, but to solve it my app actually required a before_fork and not an after_fork. Hope this helps someone.

Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }

answered Apr 07 '12 at 10:58

Paul Pettengill

4,843
1
29
33

1

This was the case for me on heroku – Bnicholas Sep 28 '12 at 20:10

score 4 · Answer 3 · answered May 09 '13 at 01:23

This article from heroku explains how to properly connect and disconnect connections for resque:

Resque.before_fork do
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.connection.disconnect!
end

Resque.after_fork do
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection
end

score 2 · Answer 4 · edited Oct 22 '12 at 12:02

2

Resque.before_fork = Proc.new { 
     ActiveRecord::ActiveRecord::Base.verify_active_connections!
}

would be even nicer

edited Oct 22 '12 at 12:02

Stephan

41,764
65
238
329

answered Oct 22 '12 at 11:41

Rob

21
1

[#verify_active_connections! was removed](https://github.com/resque/resque/issues/1098) . In rails 4 you can use ActiveRecord::Base.clear_active_connections! – MegaTux Nov 25 '15 at 02:20

Postgres error on Heroku with Resque

4 Answers4

Linked