Warp / Scotty not terminating thread / resources at end of request

Question

I'm running into an issue where my Scotty app does not seem to terminate old HTTP request threads. And eventually, after a large number (10-20) of concurrent requests, I run into an error with too many DB connections libpq: failed (FATAL: sorry, too many clients already).

{-# LANGUAGE OverloadedStrings #-}
import Web.Scotty
import Database.PostgreSQL.Simple
import Control.Monad.IO.Class

connection :: IO Connection
connection = connect defaultConnectInfo
  { connectHost = "localhost", connectUser="postgres", connectPassword="mysecretpassword" }

main :: IO ()
main = scotty 8000 $ do
  get "/" $ do
    c <- liftIO $ connection
    text "test"

This also happens with a Warp application (which Scotty):

{-# LANGUAGE OverloadedStrings #-}

import Network.Wai 
import Network.Wai.Handler.Warp (run)
import Network.HTTP.Types (status200)
import Network.HTTP.Types.Header (hContentType)
import Database.PostgreSQL.Simple
import Control.Monad.IO.Class

connection :: IO Connection
connection = connect defaultConnectInfo
  { connectHost = "localhost", connectUser="postgres", connectPassword="mysecretpassword" }

main = run 8000 app

app :: Application
app req respond = do
    respond $ responseStream status200 [] $ \write flush -> do
        print "test"
        con <- connection
        flush
        write $ "World\n"

Why is this happening? Is there an simple way to "finalize" the request at the end?

I can manually close the connection, but ideally I think killing the thread with any other related resources would be ideal.

I've verified that it keeps the connection open by running the following in postgres:

SELECT sum(numbackends) FROM pg_stat_database;

Scotty seems to take a few seconds until it closes it automatically (after the request is completed).

danidiaz · Answer 1 · 2020-01-03T16:26:33.797

postgresql-simple provides the close :: Connection -> IO () function which closes the connection and free associated resources. You need to close the connection after you are done with it.

But a common problem is the following: what happens if the code between opening and closing the connection throws an exception? How to ensure that the connection is closed even in that case, so that we don't leak connections?

In Haskell, this is solved using the bracket :: IO a -> (a -> IO b) -> (a -> IO c) -> IO c function. You pass it an IO action that allocates some resource (in this case, a connection) a function that frees the allocated resource once we are we are done with it (in our case, the close function) and a function that says what we actually want to do with the allocated resource (in this case, perform a query).

bracket is somewhat similar to try-with-resources in Java or the "using" statement in C#.

Instead of opening and closing a connection on each request, a better approach would be so use some kind of connection pool shared between request threads. persistent-postgresql uses resource-pool for example.

Warp / Scotty not terminating thread / resources at end of request

1 Answers1