0

Trying to do something similar to this question except allow underscores from the second character onwards. Not just camel case.

I can test the parser in isolation successfully but when composed in a higher level parser, I get errors

Take the following example:

#![allow(dead_code)]
#[macro_use]

extern crate nom;

use nom::*;

type Bytes<'a> = &'a [u8];

#[derive(Clone,PartialEq,Debug)]
pub enum Statement {
  IF,
  ELSE,
  ASSIGN((String)),
  IDENTIFIER(String),
  EXPRESSION,
}

fn lowerchar(input: Bytes) -> IResult<Bytes, char>{
  if input.is_empty() {
    IResult::Incomplete(Needed::Size(1))
  } else if (input[0] as char)>='a' && 'z'>=(input[0] as char) {
    IResult::Done(&input[1..], input[0] as char)
  } else {
    IResult::Error(error_code!(ErrorKind::Custom(1)))
  }
}

named!(identifier<Bytes,Statement>, map_res!(
    recognize!(do_parse!(
        lowerchar >>
        //alt_complete! is not having the effect it's supposed to so the alternatives need to be ordered from shortest to longest
        many0!(alt!(
            complete!(is_a!("_"))
            | complete!(take_while!(nom::is_alphanumeric))
            )) >> ()
        )),
        |id: Bytes| {
          //println!("{:?}",std::str::from_utf8(id).unwrap().to_string());
          Result::Ok::<Statement, &str>(
            Statement::IDENTIFIER(std::str::from_utf8(id).unwrap().to_string())
            )
        }
        ));

named!(expression<Bytes,Statement>, alt_complete!(
    identifier //=> { |e: Statement| e }
    //| assign_expr //=> { |e: Statement| e }
    | if_expr //=> { |e: Statement| e }
    ));

named!(if_expr<Bytes,Statement>, do_parse!(
    if_statement: preceded!(
      tag!("if"),
      delimited!(tag!("("),expression,tag!(")"))
      ) >>
    //if_statement: delimited!(tag!("("),tag!("hello"),tag!(")")) >>
    if_expr: expression >>
    //else_statement: opt_res!(tag!("else")) >>
    (Statement::IF)
    ));

#[cfg(test)]
mod tests {
  use super::*;
  use IResult::*;
  //use IResult::Done;

  #[test]
  fn ident_token() {
    assert_eq!(identifier(b"iden___ifiers"), Done::<Bytes, Statement>(b"" , Statement::IDENTIFIER("iden___ifiers".to_string())));
    assert_eq!(identifier(b"iden_iFIErs"), Done::<Bytes, Statement>(b"" , Statement::IDENTIFIER("iden_iFIErs".to_string())));
    assert_eq!(identifier(b"Iden_iFIErs"), Error(ErrorKind::Custom(1))); // Supposed to fail since not valid
    assert_eq!(identifier(b"_den_iFIErs"), Error(ErrorKind::Custom(1))); // Supposed to fail since not valid
  }

  #[test]
  fn if_token() {
    assert_eq!(if_expr(b"if(a)a"), Error(ErrorKind::Alt)); // Should have passed
    assert_eq!(if_expr(b"if(hello)asdas"), Error(ErrorKind::Alt)); // Should have passed
  }

  #[test]
  fn expr_parser() {
    assert_eq!(expression(b"iden___ifiers"), Done::<Bytes, Statement>(b"" , Statement::IDENTIFIER("iden___ifiers".to_string())));
    assert_eq!(expression(b"if(hello)asdas"), Error(ErrorKind::Alt)); // Should have been able to recognise an IF statement via expression parser
  }
}
candronikos
  • 159
  • 1
  • 13
  • I'm struggling to see the problem. Parsing with `expression()` seems to produce the same result as `identifier()`. – Peter Hall Jun 10 '17 at 20:00
  • 1
    A problem with your question is that it contains a lot more code than is probably required to demonstrate the issue. It would be good to strip out a lot of it, to help a potential answerer to narrow down on the issue quickly. Also, it would be easier to see what is going on if you wrote tests with failing assertions, rather than a reader having to compare the log statements. – Peter Hall Jun 10 '17 at 20:02
  • Thanks for the suggestions, Peter. I made the changes. Basically, I find that I can parse identifiers but can – candronikos Jun 11 '17 at 00:16
  • BTW You don't need the `as &[u8]` (or `as Bytes` as you changed it). The syntax `b""` already means a byte slice. – Peter Hall Jun 11 '17 at 10:45
  • None of the `if_expr` tests pass, and they are not composed in another parser. Your question is still unclear. – Peter Hall Jun 11 '17 at 14:17
  • My reply was truncated. Thanks for the suggestions, Peter. I made the changes. Basically, I find that I can parse identifiers separately but can not identify them in the condition and the expression in the if statement. I expected the expression parser nested inside to pick it up – candronikos Jun 11 '17 at 20:51
  • Changed the tests again. I commented on tests that are supposed to fail but won't throw errors when tests are run. You can see that `expr_parser` parser can detect identifiers but can't seem to do it from within the `if_token` parser. I'm not sure how that can be the case. – candronikos Jun 12 '17 at 01:55

0 Answers0