3

I have a fairly straightforward application which is running without problem locally, but when deployed to heroku it quickly stops working with a 500 error and the following error in the logs: 

ActiveRecord::ConnectionTimeoutError (could not obtain a database connection within 5.000 seconds (waited 5.003 seconds))

I have had a search around and found this question which appears to be very similar. I don't really understand from that question what the relevance of the requests for jpeg files are, and I don't understand what the probelm is from the answer given. I can recreate by restarting the server then calling a page which returns a 404 several times.

I have the same results as mentioned in the question (specifically Connections < DB_POOL): 

 (staging) $ heroku pg:info --remote staging
=== HEROKU_POSTGRESQL_ONYX_URL (DATABASE_URL)
Plan:        Hobby-dev
Status:      Available
Connections: 5
PG Version:  9.3.4
Created:     2014-05-02 08:57 UTC
Data Size:   9.9 MB
Tables:      11
Rows:        6508/10000 (In compliance)
Fork/Follow: Unsupported
Rollback:    Unsupported

(staging) $ heroku config
.....
DB_POOL:                    10
.....

I developed my solution based on this railscast which seems the same approach as in the other question. I do not however understand the answer provided in this other question and how it might apply to mine solution as follows: routes.rb:

match ':status', to: 'errors#show', constraints: {status: /\d{3}/ }, via: :all

application.rb: 

config.exceptions_app = self.routes

errorscontroller.rb

class ErrorsController < ApplicationController

    def show
        status = request.path[1..-1]
        case status
        when "404"
            @error = "404 - The page you were looking for cannot be found"
        when "403"
            @error = "403 - The page you were looking for is not accessible"
        when "500"
            @error = "500 - An error occurred within the server"
        end


        respond_to do |format|
            format.html { render action: "error" }
            format.json { render json: {status: status, error: @exception.message} }
        end
    end
end

error.html.erb

<div class='container'>
    <h1 class="errorh1">Oops something went wrong!!</h1>
    <h2 class="errorh2"><%= @error %></h2>
    <br>
    <p>We are sorry. Please <a href='#' data-uv-trigger="contact
">let us know</a> what you were trying to do so that we can look into the problem.</p>
    <br>
    <p>When you are ready to continue, <a href="/">go home</a> to start again.</p>
</div>

Note: I am running Rails 4.0.4 (and psql (PostgreSQL) 9.3.4 locally)

Community
  • 1
  • 1
Richbits
  • 7,344
  • 8
  • 33
  • 38
  • The other article is saying that a page was trying to load a MIME type that was not handled by the error controller's `respond_to`. This caused the stack to lock up. Yours is similar to his broken code. It only handles `.html` and `.json`. He fixed it by adding e.g. the `format.all { render nothing: true, status: 422 } ` cases. You could give this a try. It would also be helpful to have your `error.html.?` view code. – Gene Jul 31 '14 at 03:00
  • Thanks @Gene. I added the `format.all...` below `format.json`. I'll add the error.html.erb to the question. – Richbits Aug 02 '14 at 07:17
  • Adding `format.all` didn't fix the problem. – Richbits Aug 02 '14 at 07:28
  • Okay. That means the linked similar problem is in fact a different problem. – Gene Aug 02 '14 at 11:31
  • @Richbits are you able to do `heroku run rake db:migrate`? – Sachin Singh Aug 04 '14 at 09:31

2 Answers2

2

I dont think there is any problem with you ErrorsController, either you put it or use default one, this problem will remain there.

ActiveRecord::ConnectionTimeoutError simply means that the database has occupied all connection and it further can't accept any connection.

There are few possible reasons why this might happen. You might be doing time consuming query transaction in some action. May be you were using multithreaded server like puma or multiprocess server like unicorn (and you missed added the required configuration). There is also possibility of dead connection which keeps occupying the resources but don't lost (exactly like deadlock)

There is no key solution which will solve this problem unless you want to upgrade your database to very big plan.

I recommend you adding https://github.com/heroku/rack-timeout this will raise Racke::Timeout exception whenever the request ( internally database take longer time to respond). Either you can rescue this exception or let it keep it happen. The newrelic or airbrake might help you to give a clue where and why it is happening.

One thing i like to add in big applications, lot of time consuming sql transactions occurs too in public facing app/site but they cache all those thing. You can also do that with redis or memcache if you don't mind complexity.

The last thing, if you fail to find the solution i am afraid but be sure to contact heroku support. They are very helpful.

Paritosh Piplewar
  • 7,982
  • 5
  • 26
  • 41
  • 1
    I'm not convinced that upgrading the plan is going to help as the DB_POOL value is not exceeded in the example given. I think that I am going to have to resort to heroku support and will update here if I get a resolution. – Richbits Aug 04 '14 at 20:40
-1

Increase pool parameter in you database config.

Alex Tatarnikov
  • 353
  • 1
  • 9
  • Thanks, but I understood that if DB_POOL > connections, then increasing will make no difference. Is this an invalid understanding? – Richbits Jul 30 '14 at 20:08
  • You read this ? https://devcenter.heroku.com/articles/concurrency-and-database-connections – Alex Tatarnikov Jul 31 '14 at 06:59
  • Also it's can be a rails issue, but in 4.0.4 it should be fixed. You can read about this here http://stackoverflow.com/questions/21102115/connectiontimeouterror-on-heroku-with-postgres – Alex Tatarnikov Jul 31 '14 at 07:14
  • Thanks, I had taken a look at both of those articles, but had a little difficulty grasping the low level details being described. the first article appears to have different approaches for multi-threaded v multi process servers. Which of camps does Webbrick live in? – Richbits Aug 02 '14 at 07:38
  • 1
    @Richbits after setting `DB_POOL` have you restarted the heroku? you can restart heroku with command `heroku restart`. – Sachin Singh Aug 04 '14 at 10:19
  • @SachinSingh yes I have restarted, it is seamingly the only way to unlock it. – Richbits Aug 04 '14 at 20:36