0

I have defined my own grammar using yacc.My language gives user a flexibility to supply a extra header explicitly while invoking compiler which will be applicable to each of file to be compiled using my compiler.

I can think of two solutions to work with this scenario.

1.Append header to each of the file and compile each of the file separately.This is not a good idea as it involves editing the given source files which is necessary to append at beginning of the file.

2.Use yywrap to iterate over the list of files to be compiled and process header every time we find a new file. This is not good as it involves repeated parsing of the same file.

Without processing the header,files doesn't satisfy the grammar.

Please share the thoughts ,how it can be done in a best way ?

Ankur Gautam
  • 1,412
  • 5
  • 15
  • 27

1 Answers1

3

If your compiler is intended to compile multiple files independently, as opposed to simply concatenating them and compiling them as a single unit, and the header file is required for each input file, then you have little choice but to include it during the processing of each input file. You can do that with the yywrap mechanism, or by explicitly switching buffers using yy_create_buffer and yy_switch_to_buffer. See the flex manual for more details and sample code.

As you point out, that solution involves reparsing the header file for every input file, which might be time-consuming.

If your parse has no effect other than to create an AST, which is later processed in order to produce the compiled output or other analysis, then you could make this process more efficient by creating the AST for the header file once, and then building the ASTs for each input file by starting with a copy of the header file's AST.

You could even serialize the AST for the header file, and then read the serialized version instead of reparsing it, although then you will need to have some logic to verify that the serialized AST corresponds to the most recent version of the header file. This mechanism () is implemented in different forms by a number of C/C++ compilers (gcc, clang, Visual Studio, for example).

rici
  • 234,347
  • 28
  • 237
  • 341
  • Thanks for the input.I realize ,it can be done using yy_scan_string(char*).how ever i am still in doubt .I needs to first scan a string then a file.Will i have to maintain yy_buffer_state stack for that purpose ? and how a scan of a buffer created by a char* terminates as in case of a file it terminates with yywrap or <> ? – Ankur Gautam Jan 20 '15 at 09:30
  • It terminates with EOF. There is no need for a buffer stack unless you want one; you can just switch buffers in your EOF action. If you are going to scan the same string more than once, it is slightly more efficient to create a version of the string with two NUL terminators so that you can use yy_scan_buffer and avoid the copy. – rici Jan 20 '15 at 13:43
  • Thanks.It leads me to a new problem.Now i need to create two buffer states ,one with file* and another with char*. The problem seems is ,how can i use the size in case of using yy_create_buffer(FILE*,size_t). if let say i use YY_BUF_SIZE which is 16 k as defined or can be overwritten also but it will restrict my files to to have a limited number of character.Do i also have this YY_BUF_SIZE restriction with yyin mechanism also ? – Ankur Gautam Jan 20 '15 at 13:52
  • I also want to ask,if i use buffer stacks,does it automatically popped out when the YY_CURRENT_BUFFER reaches <> or yywrap ? – Ankur Gautam Jan 20 '15 at 13:55
  • @ankur: The size of the buffer is the amount of the file that it keeps in memory at once. Literally, the size of the buffer, not the size of the file. As recommended in the manual *which I strongly recommend you read*, you should use YY_BUF_SIZE. And, no, buffer stacks are never popped automatically. yywrap is part of the file interface, not the buffer interface. – rici Jan 20 '15 at 15:03
  • What if the size of the file is greater than the maximum possible size of the buffer ? Do i have this restriction when i am not creating any buffer explicitly ,rather using yyin to point out at the file ,i want to parse. – Ankur Gautam Jan 20 '15 at 15:50
  • @Ankur, as I said, the size of the buffer is the size of the buffer. The buffer will be refilled from the file as many times as is necessary. There is no difference between a file buffer you create and the one created by default. Flex has two types of buffers: file buffers, which are refilled as necessary, and which call yywrap when the file reaches EOF; and byte buffers, which are fixed storage areas and signal EOF at the end. – rici Jan 20 '15 at 19:29