How to do a multi instance comandline parser and checker with bison and flex for use in firmware(bare metal build enviroment)?

Question

I am playing a bit around with flex and bison in the embedded firmware creation context.

I like to create a commando line interface for my evaluation board. The commands can be send via UART or USB(ACM). That is why the parser and lexer run buffer based as two instances.

The data and processing looks like:

UART -> buffer A[10] <- lexer instance A @ main -> parser instance A -> handler

USB -> buffer B[255] <- lexer instance B @ main -> parser instance B -> handler

The buffers are filled asynchronous by the IOs. To simplify I bought everything down to a main + lexer + parser structure. There is no need for synchronization via UART and main.

My question deals with the setup of fflex and bison. How to set it up, that I can run 2 instances of them, which are buffer based?

I attach my "not working" code as simple example:

bl_lexer.c - the lexer
bl_parser.y - the parser
main.c - the main
CMakeLists.txt - cmake build

The following code:

bl_lexer.l

%option noyywrap
%option reentrant
%option prefix="bl_"

%{
#include <stdio.h>

#include "bl_parser.h"

%}

%%
[ \t]   ; // ignore all whitespace
[0-9]+      {yylval.ival = atoi(yytext); return t_int;}
\n          {return t_nl;}
"help"      {return t_help;}
"info"      {return t_info;}
%%

bl_parser.y

%{
#include <stdio.h>
#include <stdlib.h>
%}

%define api.pure full

%union {
    int ival;
}

%token<ival> t_int
%token t_nl
%token t_help
%token t_info

%start commandL

%%

info_cmd: t_info t_nl { printf("info\n");}
    ;

help_cmd: t_help t_nl { printf("help\n");}
    ;

num_cmd: t_int t_nl { printf("num: %i \n", $1);}
    ;

commandL:
    | command
    ;

command: info_cmd |
    help_cmd |
    num_cmd 
    ;

%%

main.c

#include <stdio.h>

#include "pl_parser.h"

char buf_A[10];
char buf_B[255];

int main() {
??
    return 0;
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.1)

project(bl_cmdl)

find_package(BISON REQUIRED)
find_package(FLEX REQUIRED)

set(GEN_DIR "${CMAKE_CURRENT_SOURCE_DIR}/generated")

FILE(MAKE_DIRECTORY ${GEN_DIR})

BISON_TARGET(bl_parser ${CMAKE_CURRENT_SOURCE_DIR}/bl_parser.y "${GEN_DIR}/bl_parser.c"
             DEFINES_FILE "${GEN_DIR}/bl_parser.h")

FLEX_TARGET(bl_lexer ${CMAKE_CURRENT_SOURCE_DIR}/bl_lexer.l "${GEN_DIR}/bl_lexer.c")
ADD_FLEX_BISON_DEPENDENCY(bl_lexer bl_parser)

add_executable(bl_cmdL_test
    main.c
    ${FLEX_bl_lexer_OUTPUTS}
    ${BISON_bl_parser_OUTPUTS}
)

target_include_directories(bl_cmdL_test PRIVATE ${GEN_DIR})

What embedded platform? Is this question about rapberrypi or stm32 or atmega8? — KamilCuk, Jul 10 '18 at 18:34
I am playing around with Zephyr OS on stm32 ARM. With own linker files and no stdlib — Stefan Jaritz, Jul 12 '18 at 08:42

score 1 · Answer 1 · answered Jul 11 '18 at 00:37

You've got the basic setup already, although there are a few details missing. Once you have a reentrant scanner, you can use as many instances of it as you want, since each instance has its own scanner_t object.

If you want to scan a specific buffer, you will want to call yy_scan_buffer. (That description applies to the non-reentrant interface; as indicated in the section on reentrant scanners, all APIs take the scanner_t object as an additional last argument.)

Please read the documentation of yy_scan_buffer carefully. The interface is designed to avoid copying, but it means handing your input buffer over to the flex scanner, and it will take advantage of that. There are two things you need to be aware of:

the scanner will modify the buffer, and there is no guarantee that it will be restored.
the buffer must be terminated with two NUL bytes, for obscure reasons. (yy_scan_buffer checks to make sure you've done this, in order to avoid problems later.)

(There is an annotated, fully-functional example of a reentrant parser and scanner in this answer. Below, I've just noted some important points.) Unless you use a push parser (which I personally prefer), you will need to pass the appropriate scanner_t object to the parser. The easiest way to do that is to just add it as an extra parameter to both the parser and the lexer by adding the following declaration to your bison file:

%param {scanner_t scanner}

That will cause yyparse to expect a scanner_t as an argument, and to pass it directly to yyflex.

If you are using a reentrant parser as well (which you appear to be doing, since you've asked for a pure parser), then bison will pass a pointer to yylval (and yylloc, if you're using locations) into the scanner. You'll have to make sure that the scanner is expecting those arguments, which is most easily accomplished by adding a bison-bridge declaration to your flex file.

The result of passing yylval as a pointer rather than using a global variable is that you need to change your flex actions. Instead of yylval.ival = atoi(yytext);, you'll want:

yylval->ival = atoi(yytext);

How to do a multi instance comandline parser and checker with bison and flex for use in firmware(bare metal build enviroment)?

1 Answers1