Matching template filter expressions with nom

Question

I'm working on a templating engine where some of the syntax could be like:

{{ somevar|filter }}

In place of somevar could be an arbitrary "expression", which is to say, either a variable name like somevar, or a nested filter expression (like {{ somevar|filter|anotherfilter }}). I'm trying to parse this using Rust's nom parser combinator library, but failing to get it to work so far.

Here's the parser I've come up with so far:

#[macro_use]
extern crate nom;

use std::str;

#[derive(Debug)]
pub enum Expr<'a> {
    Var(&'a [u8]),
    Filter(&'a str, Box<Expr<'a>>),
}

#[derive(Debug)]
pub enum Node<'a> {
    Lit(&'a [u8]),
    Expr(Expr<'a>),
}

named!(expr_var<Expr>, dbg_dmp!(map!(nom::alphanumeric, Expr::Var)));

named!(expr_filter<Expr>,
    dbg_dmp!(do_parse!(
         val: any_expr >>
         tag_s!("|") >>
         name: map_res!(nom::alphanumeric, str::from_utf8) >>
         (Expr::Filter(name, Box::new(val)))
    ))
);

named!(any_expr<Expr>, dbg_dmp!(ws!(
    alt_complete!(
        expr_filter |
        expr_var  
    ))));

named!(expr_node<Node>, dbg_dmp!(map!(
    delimited!(tag_s!("{{"), any_expr, tag_s!("}}")),
    Node::Expr)));

named!(parse_template< Vec<Node> >, many1!(expr_node));

With a playground. The current version panics through a stack overflow. I can fix this by reversing the expr_var | expr_filter order in any_expr, but then I'm back to basically the same error as before.

I'd strongly recommend writing out the actual grammar before trying to write the code. A strong point of parsing tools is that they help map the grammar to the code. — Shepmaster, Jan 03 '17 at 19:03
I would also recomment 1) Doing a proper http://stackoverflow.com/help/mcve (perhaps with http://play.rust-lang.org/). See also http://www.sscce.org/. 2) Providing a kind of unit test as the example of how the grammar should be parsed. — ArtemGr, Jan 04 '17 at 12:09

score 0 · Answer 1 · answered Jan 04 '17 at 11:56

I can't say I dig your question: there is no example of the text that should be parsed and neither do you describe the problem that you've encountered while building the parser.

Still, maybe the following example will be helpful. A working recursive parser:

#[macro_use]
extern crate nom;

use nom::alphanumeric;

type Variable = String;
type Filter = String;

named! (plain_expression (&str) -> (Variable, Filter), do_parse! (
    tag_s! ("{{") >>
    variable: alphanumeric >>
    tag_s! ("|") >>
    filter: alphanumeric >>
    tag_s! ("}}") >>
    ((variable.into(), filter.into()))));

#[derive(Debug)]
enum Expression {
    Plain(Variable, Filter),
    Recursive(Box<Expression>, Filter),
}

named! (recursive_expression (&str) -> Expression,
  alt_complete! (
    map! (plain_expression, |(v, f)| Expression::Plain (v, f)) |
    do_parse! (
      tag_s! ("{{") >>
      sub: recursive_expression >>
      tag_s! ("|") >>
      filter: alphanumeric >>
      tag_s! ("}}") >>
      (Expression::Recursive (Box::new (sub), filter.into())))));

fn main() {
    let plain = "{{var|fil}}";
    let recursive = "{{{{{{var1|fil1}}|fil2}}|fil3}}";
    // Prints: Done("", ("var", "fil")).
    println!("{:?}", plain_expression(plain));
    // Prints: Done("", Recursive(Recursive(Plain("var1", "fil1"), "fil2"), "fil3")).
    println!("{:?}", recursive_expression(recursive));
}

(playground).

Thanks for taking a whack at it anyway. Sorry, I wasn't sure to put the full parser code in my question, feeling that it might be overwhelming. In my defense, I linked to a GitHub repo that should have made it fairly easy to see both the full parser and the example input, as well as something that would've made it easy to test this (though maybe not as easy as your playground). Anyway, I've tried to clarify my question a bit now. I'll dig into your example a bit more to see if there's anything in there that can help. — djc, Jan 04 '17 at 19:17
@djc we don't want *the entire parser*, we want a [MCVE], emphasis on the **M**. — Shepmaster, Jan 04 '17 at 19:21
@djc Thanks for the update. Regarding the "Many1" error, you should compile nom with the "verbose-errors" feature enabled in order for it to give you the exact position of the parsing failure. — ArtemGr, Jan 04 '17 at 22:37

score 0 · Accepted Answer · answered Jan 06 '17 at 13:40

I fixed it by writing my own parser function:

named!(expr_var<Expr>, map!(nom::alphanumeric, Expr::Var));

fn expr_filtered(i: &[u8]) -> IResult<&[u8], Expr> {
    let (mut left, mut expr) = match expr_var(i) {
        IResult::Error(err) => { return IResult::Error(err); },
        IResult::Incomplete(needed) => { return IResult::Incomplete(needed); },
        IResult::Done(left, res) => (left, res),
    };
    while left[0] == b'|' {
        match nom::alphanumeric(&left[1..]) {
            IResult::Error(err) => {
                return IResult::Error(err);
            },
            IResult::Incomplete(needed) => {
                return IResult::Incomplete(needed);
            },
            IResult::Done(new_left, res) => {
                left = new_left;
                expr = Expr::Filter(str::from_utf8(res).unwrap(), Box::new(expr));
            },
        };
    }
    return IResult::Done(left, expr);
}

named!(expr_node<Node>, map!(
    delimited!(tag_s!("{{"), ws!(expr_filtered), tag_s!("}}")),
Node::Expr));

There is probably some nicer way to do the same thing with nom macros, but at least I got something working.

Matching template filter expressions with nom

2 Answers2