15.5.18.1 General Problems Parsing Perl Code

It is often heard that only Perl can parse Perl. This is not true. Perl cannot be parsed at all, it can only be executed. Perl has various built-in ambiguities that can only be resolved at runtime.

The following example may illustrate one common problem:

     print gettext "Hello World!";

Although this example looks like a bullet-proof case of a function invocation, it is not:

     open gettext, ">testfile" or die;
     print gettext "Hello world!"

In this context, the string gettext looks more like a file handle. But not necessarily:

     use Locale::Messages qw (:libintl_h);
     open gettext ">testfile" or die;
     print gettext "Hello world!";

Now, the file is probably syntactically incorrect, provided that the module Locale::Messages found first in the Perl include path exports a function gettext. But what if the module Locale::Messages really looks like this?

     use vars qw (*gettext);
     
     1;

In this case, the string gettext will be interpreted as a file handle again, and the above example will create a file testfile and write the string “Hello world!” into it. Even advanced control flow analysis will not really help:

     if (0.5 < rand) {
        eval "use Sane";
     } else {
        eval "use InSane";
     }
     print gettext "Hello world!";

If the module Sane exports a function gettext that does what we expect, and the module InSane opens a file for writing and associates the handle gettext with this output stream, we are clueless again about what will happen at runtime. It is completely unpredictable. The truth is that Perl has so many ways to fill its symbol table at runtime that it is impossible to interpret a particular piece of code without executing it.

Of course, xgettext will not execute your Perl sources while scanning for translatable strings, but rather use heuristics in order to guess what you meant.

Another problem is the ambiguity of the slash and the question mark. Their interpretation depends on the context:

     # A pattern match.
     print "OK\n" if /foobar/;
     
     # A division.
     print 1 / 2;
     
     # Another pattern match.
     print "OK\n" if ?foobar?;
     
     # Conditional.
     print $x ? "foo" : "bar";

The slash may either act as the division operator or introduce a pattern match, whereas the question mark may act as the ternary conditional operator or as a pattern match, too. Other programming languages like awk present similar problems, but the consequences of a misinterpretation are particularly nasty with Perl sources. In awk for instance, a statement can never exceed one line and the parser can recover from a parsing error at the next newline and interpret the rest of the input stream correctly. Perl is different, as a pattern match is terminated by the next appearance of the delimiter (the slash or the question mark) in the input stream, regardless of the semantic context. If a slash is really a division sign but mis-interpreted as a pattern match, the rest of the input file is most probably parsed incorrectly.

If you find that xgettext fails to extract strings from portions of your sources, you should therefore look out for slashes and/or question marks preceding these sections. You may have come across a bug in xgettext's Perl parser (and of course you should report that bug). In the meantime you should consider to reformulate your code in a manner less challenging to xgettext.