Bison C++ get name of token - yytname_ is private

Question

I'm attempting to get the name of a token in C++ Bison: E.g. %token <int> TPLUS "+" TMINUS "-" TMUL "*" TDIV "/"

However in the C++ variant of Bison, %token-table does not do anything.

I have noticed that there is a token mapping in the generated bison code: const char* const parser::yytname_[]; however it is private.

Does anyone have any idea of how to get the names of these tokens in C++ Bison? Is this even possible using the C++ variant?

I know this question was asked here, but there is no accepted answer.

Using version 3.4.3

My bison manual doesn't mention any C++ restriction for %token-table. If you're sure it doesn't do anything, it might be worthwhile asking the project what the restriction is, and how you're supposed to use it. From the documentation, I would expect it produce the same global array it does under C. — James K. Lowden, Jun 17 '21 at 17:36
Doesn't [`symbol_name`](https://www.gnu.org/software/bison/manual/bison.html#index-symbol_005fname-on-parser) give you what you need? — rici, Jun 17 '21 at 18:52
@Tom: I think you need quite a recent version of Bison, preferably the one which corresponds to that documentation (3.7.6). I don't remember exactly when that interface was added, but it wasn't long ago. — rici, Jun 17 '21 at 19:12
Also, you need to %define parse.error to custom, detailed or verbose (and verbose changes the return type of symbol_name). — rici, Jun 17 '21 at 19:22
@rici Upgrading Bison to the latest version solved the problem. Many thanks. — Tom, Jun 18 '21 at 00:20
Cool. In the meantime, I wrote up an answer with what I discovered looiking through the various C++ templates in the bison source. — rici, Jun 18 '21 at 01:05

score 1 · Accepted Answer · answered Jun 18 '21 at 01:00

First, ensure that you have a recent version of Bison (I believe the minimum is v3.6, but the v3.7 versions have several useful bug fixes).

That will generate a static member function named symbol_name with one of the following prototypes. Note that `token_symbol_kind is an internal token number, not the value produced by the scanner. (See below)

const char* yy::parser::symbol_name(token_symbol_kind)  (1)
std::string yy::parser::symbol_name(token_symbol_kind)  (2)

(1) if %define parse.error custom|detailed
(2) if %define parse.error verbose

If the option parse.error is not defined, then you can still get the symbol_name definition by using the deprecated %token-table directive, or if you arrange for #define YYDEBUG to be inserted in the generated code (see the -d flag, for example). I have no idea why the return type differs depending on the definition of parse.error.

As noted, the argument to symbol_name is a bison internal token number, rather than the token type returned by the scanner. You can get the internal token number for a token type using the parser member class by_kind, with an expression like yy::parser::by_kind(yy::parser::token::«token-type-name»).type_get().

score 0 · Answer 2 · answered Jun 18 '21 at 00:39

Make sure you're bison is the latest version (3.7.3) in my case. The name of a token can be derived by:

std::cout << yy::parser::symbol_name(yy::parser::by_kind(yy::parser::token::TPLUS).type_get()) << std::endl;

Where y::parser::token::TPLUS is your token.

Bison C++ get name of token - yytname_ is private

2 Answers2

Linked