1

I'm trying to find a way to check if a webpage exists in Haskell. The server is HTTP2 / HTTPS only and I'm trying to check if the page exists in a servant application.

Is there any Haskell packages with good documentation to just check if the status code is 200 or 404? And working with strong HTTPS and HTTP2 servers?

Here what I currently have with http-conduit but I'm receiving weird exceptions (TlsExceptionHostPort (HandshakeFailed (Error_Protocol ("expecting server hello, got alert : [(AlertLevel_Fatal,HandshakeFailure)]",True,HandshakeFailure))) "thibaud.dauce.fr" 443 and StatusCodeException).

... other imports
import qualified Network.HTTP.Conduit as HTTP

... other types
type AppM = ReaderT Config (EitherT ServantErr IO)

newComment :: String -> OneComment -> AppM Int64
newComment baseUrl oneComment = do
    time <- liftIO getCurrentTime
    response <- HTTP.withManager $ \manager -> do
        request <- HTTP.parseUrl $ url oneComment
        HTTP.httpLbs request manager
    case (statusIsSuccessful $ HTTP.responseStatus response, startswith baseUrl (url oneComment)) of
        (_, False) -> return 0
        (True, True) -> do
            theNewComment <- runDb $ insert $ Comment (url oneComment) (content oneComment) time
            return $ fromSqlKey theNewComment
        _ -> return 0
Thibaud Dauce
  • 367
  • 4
  • 14

1 Answers1

3

Some examples using wreq

{-# LANGUAGE OverloadedStrings #-}

import Network.Wreq
import Control.Lens
import Control.Exception as E
import Network.HTTP.Client (HttpException)

test1 = do
  r <- get "https://httpbin.org/get"
  print $ r ^. responseStatus . statusCode

-- throws an exception
test2 = do
  r <- get "https://www.google123123.com"
  print $ r ^. responseStatus . statusCode

testUrl url = do
  r <- get url
  return $ r ^. responseStatus . statusCode

-- catching the exception
test3 = do
  st <- testUrl "https://www.google123123123.com"  `E.catch` handler
  print st
  where
    handler :: HttpException -> IO Int
    handler _ = return 999
ErikR
  • 51,541
  • 9
  • 73
  • 124
  • I use your code and I get a `TlsExceptionHostPort (HandshakeFailed (Error_Protocol ("expecting server hello, got alert : [(AlertLevel_Fatal,HandshakeFailure)]",True,HandshakeFailure))) "thibaud.dauce.fr" 443` even if my website as a perfectly valid HTTPS certificate… – Thibaud Dauce May 08 '16 at 14:47
  • `wreq` uses [`hs-tls`](https://github.com/vincenthz/hs-tls) and there have been issues with certain kinds of certificates and ciphers. I would build the latest version from the github repo and run the `tis-simpleclient` program to get more diagnostic info. And also check the issue tracker - there have been problems with specific ISP providers. – ErikR May 08 '16 at 15:09
  • Already publish an issue to have more information https://github.com/vincenthz/hs-tls/issues/142 a few days ago. `tls-simpleclient` seems to fail too… Thanks for your help and for your code samples again! – Thibaud Dauce May 08 '16 at 15:45