Parslet: How to Buffer/parse incrementally data

Question

I'm writing an HTTP/1 response parser with parslet. It works, but only when I send the full payload.

I have smth like this:

rule(:response) {
   response_line >> crlf >>
   header.repeat.as(:headers) >> crlf >>
   data.as(:data)
}
root :response

But if I pass an incomplete payload, I get:

parser.parse("HTTP/1.1 200 OK\r\n")
#=> Parslet::ParseFailed: Failed to match sequence (RESPONSE_LINE CRLF headers:(HEADER{0, }) CRLF data:DATA) at line 1 char 16.

I'd like to be able to feed bytes to the parser without failing, at least if they don't break the expectations. Is there a way to somehow "buffer" until some rule is broken, or all expectations are met?

does this implementation not provide what you need? https://github.com/postmodern/net-http-server/blob/master/lib/net/http/server/parser.rb — lacostenycoder, Nov 06 '18 at 05:54
No, and I've seen this parser. If the header frame is chunked into different parts, it will trigger a parsing error, exactly because it doesn't buffer. — ChuckE, Nov 06 '18 at 08:31

score 0 · Answer 1 · answered Dec 04 '18 at 23:12

Parslet matches a grammer against a whole document.

If you want to allow it to parse a partial document you need to define your grammar such that the missing parts are optional.

One approach may be to define a grammar that would match any one elements from the header, and define a 'the_rest' capture group that matches 'any.repeat'

Then you could recursively call the parser each time you get more document... with "the rest" plus anything more you have read in.

Each time you call it you would get one part of the header returned.

Parslet: How to Buffer/parse incrementally data

1 Answers1