Conditional lookahead in attoparsec

Question

Assume there is a data structure representing a text with comments inside.

data TWC
  = T Text TWC -- text
  | C Text TWC -- comment
  | E -- end
  deriving Show

Thus string like

"Text, {-comment-}, and something else"

could be encoded as

T "Text, " (C "comment" (T ", and something else" E))

Parsers for comment chunk and for E are pretty trivial:

twcP :: Parser TWC
twcP = eP <|> cP <|> tP

cP :: Parser TWC
cP = do
  _ <- string "{-"
  c <- manyTill anyChar (string "-}")
  rest <- cP <|> tP <|> eP
  return (C (pack c) rest)

eP :: Parser TWC
eP = do
  endOfInput
  return E

Implementing parser for text chunk in such a trivial way

tP :: Parser TWC
tP = do
  t <- many1 anyChar
  rest <- cP <|> eP
  return (T (pack t) rest)

make it consume comments section as a text because of its greedy nature

> parseOnly twcP "text{-comment-}"
Right (T "text{-comment-}" E)
it ∷ Either String TWC

So, the question is how to express the logic of parsing until end of input or until comment section? In other words, how to implement conditional lookahead parser?

score 5 · Accepted Answer · answered Jul 05 '16 at 14:15

You're right, the problematic code is the first line of of tP, which parses text greedily without stopping at comments:

tP = do
  t <- many1 anyChar

Before addressing that, I first want to refactor your code a little to introduce helpers and use applicative style, with the problematic code isolated into the text helper:

-- Like manyTill, but pack the result to Text.
textTill :: Alternative f => f Char -> f b -> f Text
textTill p end = pack <$> manyTill p end

-- Parse one comment string
comment :: Parser Text
comment = string "{-" *> textTill anyChar (string "-}")

-- Parse one non-comment text string (problematic implementation)
text :: Parser Text
text = pack <$> many1 anyChar

-- TWC parsers:

twcP :: Parser TWC
twcP = eP <|> cP <|> tP

cP :: Parser TWC
cP = C <$> comment <*> twcP

eP :: Parser TWC
eP = E <$ endOfInput

tP :: Parser TWC
tP = T <$> text <*> twcP

To implement lookahead, we can use the lookAhead combinator, which applies a parser without consuming the input. That allows us to make text parse until it reaches either a comment (without consuming it), or endOfInput:

-- Parse one non-comment text string (working implementation)
text :: Parser Text
text = textTill anyChar (void (lookAhead comment) <|> endOfInput)

With that implementation, twcP behaves as expected:

ghci> parseOnly twcP "text{-comment-} post"
Right (T "text" (C "comment" (T " post" E)))

Conditional lookahead in attoparsec

1 Answers1