Why does this parser always fail when the end-of-line sequence is CRLF?

Question

This simple parser is expected to parse messages of the form

key: value\r\nkey: value\r\n\r\nkey: value\r\nkey: value\r\n\r\n

One EOL acts as a field separator, and double EOL acts as a message separator. It works perfectly fine when the EOL separator is \n but parseWith always returns fail when it is \r\n.

parsePair = do
  key <- B8.takeTill (==':')
  _ <- B8.char ':'
  _ <- B8.char ' '
  value <- B8.manyTill B8.anyChar endOfLine
  return (key, value)

parseListPairs = sepBy parsePair endOfLine <* endOfLine

parseMsg = sepBy parseListPairs endOfLine <* endOfLine

Answer 1

I'm assuming you are using these imports:

{-# LANGUAGE OverloadedStrings #-}
import qualified Data.Attoparsec.ByteString.Char8 as B8
import Data.Attoparsec.ByteString.Char8

The problem is that endOfLine consumes the end of line, so perhaps you really want something like:

parseListPairs = B8.many1 parsePair <* endOfInput

For instance, this works:

ghci> parseOnly parseListPairs "k: v\r\nk2: v2\r\n"
Right [("k","v"),("k2","v2")]

Update:

For parsing multiple messages you can use:

parseListPairs = B8.manyTill parsePair endOfLine
parseMsgs = B8.manyTill parseListPairs endOfInput

ghci> test3 = parseOnly parseMsgs "k1: v1\r\nk2: v2\r\n\r\nk3: v3\r\nk4: v4\r\n\r\n"
Right [[("k1","v1"),("k2","v2")],[("k3","v3"),("k4","v4")]]

Answer 2

Problems

Your code isn't self-contained and the actual problem is unclear. However, I suspect your woes are actually caused by how keys are parsed; in particular, something like \r\nk is a valid key, according to your parser:

λ> parseOnly parsePair "\r\nk: v\r\n"
Right ("\r\nk","v")

That needs to be fixed.

Moreover, since one EOL separates (rather than terminates) key-value pairs, an EOL shouldn't be consumed at the end of your parsePair parser.

Another tangential issue: because you use the many1 combinator instead ByteString-oriented parsers (such as takeTill), your values have type String instead of ByteString. That's probably not what you want, here, because it defeats the purpose of using ByteString in the first place.; see Performance considerations.

Solution

I suggest the following refactoring:

{-# LANGUAGE OverloadedStrings #-}

import Data.ByteString ( ByteString )

import Data.Attoparsec.ByteString.Char8 ( Parser
                                        , count
                                        , endOfLine
                                        , parseOnly
                                        , sepBy
                                        , string
                                        , takeTill
                                        )

-- convenient type synonyms
type KVPair = (ByteString, ByteString)
type Msg = [KVPair]

pair :: Parser KVPair
pair = do
    k <- key
    _ <- string ": "
    v <- value
    return (k, v)
  where
    key     = takeTill (\c -> c == ':' || isEOL c)
    value   = takeTill isEOL
    isEOL c = c == '\n' || c == '\r'

-- one EOL separates key-value pairs
msg :: Parser Msg
msg = sepBy pair endOfLine

-- two EOLs separate messages
msgs :: Parser [Msg]
msgs = sepBy msg (count 2 endOfLine)

I have renamed your parsers, for consistency with attoparsec's, none of which have "parse" as a prefix:

• parsePair --> pair
• parseListPairs --> msg
• parseMsg --> msgs

Tests in GHCi

λ> parseOnly keyValuePair  "\r\nk: v"
Left "string"

Good; you do want a fail, in this case.

λ> parseOnly keyValuePair  "k: v"
Right ("k","v")

λ> parseOnly msg "k: v\r\nk2: v2\r\n"
Right [("k","v"),("k2","v2")]

λ> parseOnly msgs  "k1: v1\r\nk2: v2\r\n\r\nk3: v3\r\nk4: v4"
Right [[("k1","v1"),("k2","v2")],[("k3","v3"),("k4","v4")]]

λ> parseOnly msgs "k: v"
Right [[("k","v")]]