13.2 How Can I Reset the Parser

The following phenomenon has several symptoms, resulting in the following typical questions:

I invoke yyparse several times, and on correct input it works properly; but when a parse error is found, all the other calls fail too. How can I reset the error flag of yyparse?

or

My parser includes support for an ‘#include’-like feature, in which case I run yyparse from yyparse. This fails although I did specify ‘%define api.pure full’.

These problems typically come not from Bison itself, but from Lex-generated scanners. Because these scanners use large buffers for speed, they might not notice a change of input file. As a demonstration, consider the following source file, first-line.l:

%{
#include <stdio.h>
#include <stdlib.h>
%}

%%
.*\n    ECHO; return 1;
%%

int
yyparse (char const *file)
{
  yyin = fopen (file, "r");
  if (!yyin)
    {
      perror ("fopen");
      exit (EXIT_FAILURE);
    }

  /* One token only. */
  yylex ();
  if (fclose (yyin) != 0)
    {
      perror ("fclose");
      exit (EXIT_FAILURE);
    }
  return 0;
}

int
main (void)
{
  yyparse ("input");
  yyparse ("input");
  return 0;
}

If the file input contains

input:1: Hello,
input:2: World!

then instead of getting the first line twice, you get:

$ flex -ofirst-line.c first-line.l
$ gcc  -ofirst-line   first-line.c -ll
$ ./first-line
input:1: Hello,
input:2: World!

Therefore, whenever you change yyin, you must tell the Lex-generated scanner to discard its current buffer and switch to the new one. This depends upon your implementation of Lex; see its documentation for more. For Flex, it suffices to call ‘YY_FLUSH_BUFFER’ after each change to yyin. If your Flex-generated scanner needs to read from several input streams to handle features like include files, you might consider using Flex functions like ‘yy_switch_to_buffer’ that manipulate multiple input buffers.

If your Flex-generated scanner uses start conditions (see Start conditions in The Flex Manual), you might also want to reset the scanner’s state, i.e., go back to the initial start condition, through a call to ‘BEGIN (0)’.