JSON parsing using Lemon (and Core Foundation)

Question

I'm trying to write a simple JSON parser using Lemon and Apple Core Foundation.

Here's the code so far:

%include {

#import <CoreFoundation/CoreFoundation.h>

#import "state.h" // struct ParserState { CFTypeRef result; };
#import "tuple.h" // struct Tuple { CFTypeRef one; CFTypeRef two; };

}

%start_symbol json

%token_type { CFTypeRef }
%token_prefix T

%extra_argument  { ParserStateRef state }

%type simple_value { CFTypeRef }
%type member { TupleRef }
%type members { CFMutableDictionaryRef }
%type object { CFMutableDictionaryRef }
%type array { CFMutableArrayRef }

simple_value(A) ::= STRING(B). { A = B; }
simple_value(A) ::= INT(B). { A = B; }
simple_value(A) ::= FLOAT(B). { A = B; }
simple_value(A) ::= FALSE. { A = kCFBooleanFalse; }
simple_value(A) ::= TRUE. { A = kCFBooleanTrue; }
simple_value(A) ::= NULL. { A = kCFNull; }

member(A) ::= STRING(B) COLON simple_value(C). {
    A = TupleCreate(B,C);
}
member ::= STRING COLON object.
member ::= STRING COLON array.

members(A) ::= member(B). {
    A = CFDictionaryCreateMutable(kCFAllocatorDefault,0,&kCFTypeDictionaryKeyCallBacks,&kCFTypeDictionaryValueCallBacks);
    CFDictionarySetValue(A, B->first, B->second);
    CFRelease(B->first);
    CFRelease(B->second);
    TupleRelease(B);
}
members(A) ::= members(B) COMMA member(C). {
    CFDictionarySetValue(B, C->first, C->second);
    CFRelease(C->first);
    CFRelease(C->second);
    TupleRelease(C);
    A = B;
}

values ::= value.
values ::= values COMMA value.

object(A) ::= LCB RCB. {
/* THIS NEVER GETS CALLED */
    A = CFDictionaryCreateMutable(kCFAllocatorDefault,0,&kCFTypeDictionaryKeyCallBacks,&kCFTypeDictionaryValueCallBacks);
}
object(A) ::= LCB members(B) RCB. {
/* THIS NEVER GETS CALLED */
    A = B;
}

array ::= LSB RSB.
array ::= LSB values RSB.

value ::= array.
value ::= object.
value ::= simple_value.

json ::= object(A). { state->result = A; }
json ::= array.

With a simple JSON like this

{ \"hello\" : \"world\" }

I can't go past the members rule (at that point, the dictionary is set up correctly).

The object rule is never called, and the json ::= object does the same!

Am I doing something stupid?

Any input will be appreciated!

score 1 · Answer 1 · answered Dec 09 '15 at 17:48

You have to call Parser(...) function with zero value for second parameter when tokens stream is run out.

Information from: https://www.sqlite.org/src/doc/trunk/doc/lemon.html

   01 ParseTree *ParseFile(const char *zFilename){
   02    Tokenizer *pTokenizer;
   03    void *pParser;
   04    Token sToken;
   05    int hTokenId;
   06    ParserState sState;
   07
   08    pTokenizer = TokenizerCreate(zFilename);
   09    pParser = ParseAlloc( malloc );
   10    InitParserState(&sState);
   11    while( GetNextToken(pTokenizer, &hTokenId, &sToken) ){
   12       Parse(pParser, hTokenId, sToken, &sState);
   13    }
   14    Parse(pParser, 0, sToken, &sState);
   15    ParseFree(pParser, free );
   16    TokenizerFree(pTokenizer);
   17    return sState.treeRoot;
   18 }

See line 14. It says to parser that it should not expect more tokens and can perform remained rules.

Basically, what a program has to do to use a Lemon-generated parser is first create the parser, then send it lots of tokens obtained by tokenizing an input source. When the end of input is reached, the Parse() routine should be called one last time with a token type of 0. This step is necessary to inform the parser that the end of input has been reached. Finally, we reclaim memory used by the parser by calling ParseFree().

JSON parsing using Lemon (and Core Foundation)

1 Answers1