This manual (updated 28 June 2014) is for GNUnited Nations (version
0.9), a suite for maintaining translations of www.gnu.org
essays and other articles.
Copyright © 2008, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License.”
|• Introduction:||Overview of GNUnited Nations.|
|• Usage:||Basic usage, invocation and tips.|
|• Team's Repository:||How to maintain translations in the team’s repository.|
|• PO Files:||The gentle art of editing PO files.|
|• Webmaster Tips:||The webmaster’s guide to GNUnited Nations’ galaxy.|
|• Internals:||Dive into GNUN.|
|• Bugs:||How to report bugs.|
|• Copying This Manual:||The GNU Free Documentation License.|
GNUnited Nations (abbreviated GNUN) is a collection of scripts that are supposed to make the life of https://www.gnu.org translators easier. Although it is specifically developed for the GNU Project’s website, it could be customized, at least in theory, to fit the needs of other internationalized sites. GNUN is in an early stage of development, but if it proves useful, and if there is sufficient interest (and time), it is possible to develop a robust configuration interface that would be appropriate for general usage.
It is vitally important to understand that GNUN is not a silver bullet that solves all problems. To be honest, deploying GNUN in fact even does create some Disadvantages.
GNUnited Nations is free software, available under the GNU General Public License.
This manual is organized in a way that is suitable both for translators and GNU Web Translation Managers (plus eventually interested GNU Webmasters, if any). It may also serve as an introductory material and reference for new GNUN developers and contributors. Hopefully, it might be useful to people who customize and adopt the software for a third party site or for their own needs. Feel free to skip sections or entire chapters if they are irrelevant for your intended usage.
This manual is free documentation, and you can modify and redistribute it under the terms of the GNU Free Documentation License. See GNU Free Documentation License.
|• Overview:||What is GNUN and why is necessary?|
|• Concepts:||Basic concepts and goals.|
|• Advantages:||The goodness GNUN brings.|
|• Disadvantages:||Staying on firm ground.|
The GNU Project’s website, has become considerably large over the years. Maintaining it requires significant effort, and sometimes a new web standard is developed faster than the time required to migrate all articles to the next widely adopted one.
When it comes to internationalization, the problems are so many that it is hard to enumerate them. It has become apparent that maintaining translations up-to-date is a major undertaking, involving tedious skimming through commit logs, reviewing diffs and other medieval techniques to catch up. Some translation teams have developed their own sets of scripts, but so far there has been no universal solution.
This unpleasant situation, combined with rapid and incompatible design changes, have led some teams to neglect the important work of keeping their translations in line with the changing original articles. As a consequence, the GNU Project is facing the problem of maintaining them in suboptimal ways, in order to keep the information updated.
The reasons for developing GNUnited Nations are very similar to those
that led to the inception of GNU gettext, or GNOME Documentation
gnome-doc-utils) some years later.
The basic concept behind GNUN is that localization of HTML articles is similar to localization of computer programs1. In articles, like in programs, not every string is considered translatable, so translatable strings must be identified first, and then collected in a file (called “PO template”) for translation. Articles, like programs, tend to change over time, but not every change in the sources calls for a translation update. Sometimes the change does not affect the translatable strings, but sometimes it does. So, translators must have means to identify those changes and apply the appropriate updates to the translation.
gettext package already provides the needed
infrastructure for maintaining translations using PO files.
See Introduction in GNU gettext tools,
for a basic overview.
GNUnited Nations fills the gaps to apply this infrastructure to articles
in the https://www.gnu.org web site.2
The following diagram summarizes the relation between the files handled by GNUN. It is followed by somewhat detailed explanations, which you should read while keeping an eye on the diagram. Having a clear understanding of these interrelations will surely help translators and web maintainers.
.---<--- * Original ARTICLE.html | | .---> ARTICLE.pot ---> * ARTICLE.LANG.po --->---. `---+ | `--->---. .------<----------------------------' | | | `---. | +---> Translated ARTICLE.LANG.html `-------'
The indication ‘*’ appears in two places in this picture, and means that the corresponding file is intended to be edited by humans. The author or web maintainer edits the original article.html, and translators edit article.lang.po. All other files are regenerated by GNUN and any manual changes on them will be lost on the next run.
Arrows denote dependency relation between files, where a change in one
file will affect the other. Those automatic changes will be applied by
running ‘make -C server/gnun’. This is the primary way to invoke
GNUN, since it is implemented as a set of recipes for GNU
First, GNUN extracts all translatable strings from the original English
article article.html into
resulting file is suitable for manipulation with the various GNU
‘gettext’ utilities. It contains all original article strings and
all translations are set to empty. The letter
t in .pot
marks this as a Template PO file, not yet oriented towards any
On the first run, though, there is no article.lang.po yet, so a translator must create article.lang.po from article.pot, where lang represents the target language. See New Translation, for details.
Then comes the initial translation of messages in article.lang.po. Translation in itself is a whole matter, whose complexity far overwhelms the level of this manual. Nevertheless, a few hints are given in another chapter of this manual.
It is possible to make GNUN get translations for common strings from dedicated PO files, the so-called compendia. See Compendia, for more information.
You may use any compatible PO editor to add translated messages into the PO file. See Editing in GNU gettext tools, for more information.
When the PO file actually exists (hopefully populated with initial translations), GNUN generates article.lang.html file. It takes its structure from the original article.html, but all translatable strings are replaced with the translations specified in article.lang.po.
Original articles sometimes change. A new paragraph is being added or a tiny change in the wording is introduced. Also, some articles are dynamic in nature, like the ones containing news entries or a list of other articles. If the original article changes, GNUN will automatically rebuild article.pot, and will merge the changes into article.lang.po. Any outdated translations will be marked as “fuzzy,” any new strings will be added with empty translations, waiting to be translated. In the same run article.lang.html will be rebuilt, and consequently the relevant strings in the translation will be substituted with the original English text until the translation teams update them in article.lang.po.
Those changes in the original article that do not affect the translatable strings (or just delete whole strings) will not lead to new untranslated or “fuzzy” strings in article.lang.po. Thus, no actions from translators will be needed. article.lang.html will be automatically regenerated to reflect the changes.
The POT for every article under GNUN’s control is kept in the ‘www’
repository under a special sub-directory po/ of the relevant
directory in the ‘www’ tree. Thus, for
sub-directory is philosophy/po/. In addition to
free-sw.pot, this directory holds the canonical source of every
translation, such as free-sw.bg.po, free-sw.ca.po, etc.
For more details, see Files and Directories.
Several additional features are implemented, like automatic update of the list of the available translations. For example, if a new free-sw.ja.po translation is added, the list of translations included in free-sw.html and all translated free-sw.lang.html is updated. This saves a lot of tedious, repetitive work and eliminates a source of mistakes. There is a basic infrastructure to “inject” general information about a translation team—like a note on how to contact the team, or how to report a bug/suggestion for improvement. Translators’ credits are also handled, as well as translators’ notes, if any.
Here is a simple list of situations where we hope this suite would prove useful.
As it happens in real life, we don’t wear pink glasses and are aware of certain limitations and annoyances of this semi-automatic system.
If anything may go wrong, it will definitely go wrong. —Murphy’s Law
Murphy is an optimist. —O’Reilly’s Law
GNUN currently consists of a few makefiles, scripts and template files.
They are designed
to reside in the server/gnun directory, but this may change.
In all examples in this manual, “invoking” means executing on the
make -C server/gnun [target]
[variable=value …] while the working directory is
the root in the ‘www’ web repository. For the purpose of brevity, we
will refer to the above command as simply
make, which is
cd server/gnun; make. It is desirable never to
make with the -k (--keep-going)
option, because an eventual error in only one make recipe might create
a mess in many articles, both original and translated. Do this with
caution, and generally only when debugging in a safe environment.
The build process is intended to be invoked by a cron job, although manual intervention is possible to a certain degree.
|• Invoking GNUN:||How to trigger a (re)build.|
|• Runtime Variables:||Variables to control the build process.|
|• Special Targets:||Targets that are not built by default.|
|• Main Variables:||Specifying what to build.|
|• languages.txt:||Specifying canonical names for languages.|
|• Compendia:||Using translation memory.|
|• Sitemap:||Specifics of sitemap generation.|
The central part of GNUnited Nations is a makefile; actually a
GNUmakefile since it heavily relies on features and extensions
available in GNU Make. Thus, invoking a build consists of typing
make on the command line, or within cron. If you are
deploying the software on a non-GNU machine, probably GNU Make is
installed and available as
not, you should consider installing it, since the build will fail
otherwise. See GNU Make homepage for information on
how to download and install GNU Make.
If you don’t specify a target,
make by default builds the
all, which in this case is to rebuild all translations
that are not up-to-date. However, there are special targets that do not
depend on the standard
all target, which can be built by
make target. Some of the variables in the next section
apply to them, and some do not.
Note that GNUN expects GNUmakefile, config.mk and
gnun.mk to be present under server/gnun of the
‘www’ web repository, otherwise
make has no way to know what
to built, and how. Another file, priorities.mk, is
expected to be present under server/gnun in order to define
the priorities when reporting about the outdated translations. If
report target will not sort the translations by
priority. Since the location of the repository working
copy is strictly user-specific and cannot be determined in any way,
GNUmakefile and config.mk must be copied there manually
after the package installation. For convenience, these files are
installed in ‘$(pkgdatadadir)’ (/usr/local/share/gnun with
the default ‘prefix’) so you can just create symlinks pointing to
ln -s /usr/local/share/gnun/config.mk /path/to/www/server/gnun/ ln -s /usr/local/share/gnun/GNUmakefile /path/to/www/server/gnun/
If next GNUN releases are installed with the same --prefix, you will always use the latest versions without the need for any manual intervention.
If you are playing with a non-gnu.org setup, you also have to take care of gnun.mk and put a modified version under server/gnun of your hypothetical tree. (For gnu.org, that is not necessary since a proper gnun.mk is maintained in ‘www’.)
The build process has several modes of operation, and they all relate to
the handling of files that are to be added to the repository or
performing certain sanity checks at build time. The variables are
specified on the command line, after
make, in the form
make VCS=yes. In the future,
additional features will be implemented in a similar fashion.
Do not add any files to the repository. This is the default. You may
as well omit to define
VCS entirely; there is no special code
that expects assigning the value ‘no’.
Automatically add any new files in the repository (CVS, Subversion or
GNU Bazaar—the repository type is auto-determined at build time,
bzr being a fallback).4 These are
any POT files, if they are generated for the first time, and the
translated articles (.lang.html) in HTML format. Finally,
any missing PO and their HTML counterparts of
the server templates will be added, computed on the basis of the
Skips validation of the HTML articles and generated translations.
Validates all original articles before generating the POTs, to ensure that the ultimate source is valid (X)HMTL. Also, validates all generated translations in HTML format and all PO files. It is highly recommended to run the build this way, even if it is a bit tedious to fix the errors that are reported as a result of enforcing validation.
Articles defined in the variable
no-validate-articles are never
checked for HTML validity. Its purpose was to skip validation of HTML5
articles, until the command-line validation tools are updated to
parse files that comply with this new standard. As of version 0.5
GNUN supports HTML5, so resorting to
should not be necessary.
This is the default, and not defining this variable has the same effect.
Do not send email notifications about errors. This is the default.
If an error occurs, send a mail with a meaningful subject and the error
message as body to the concerned party. The variables
transl-addr control the
recipients; normally they should be set to the GNUN maintainers,
webmasters and translators accordingly. These variables are defined in
the configuration file gnun.conf and by default are set to
If defined, automatic announcements for new translations will be sent to
the address defined in the
ann-addr variable (in
gnun.conf). The email messages contain the translated article
title as Subject, and the URL of the translation as its body. For the
official GNUN build, they are delivered to the
email@example.com mailing list and each language has its
own Mailman topic. It is possible to subscribe to the list and
receive only traffic related to a specific language. See Mailing
Lists in GNU Web Translators Manual.
The default behavior is not to send such announcements.
If defined, the value of the variables
will be printed to the standard output. This is off by
default, but recommended in general since it will show a bug in the
computation of the basic variables.
If defined, ordinary articles that have “fuzzy” strings and are not older
than days will not be regenerated. This functionality is
implemented specifically to prevent gratuitous replacement of translated
strings with the English text when there are only minor formatting
changes in the original. The translator has time (the “grace” period
as defined in this variable) to review the changes and clear the “fuzzy”
mark from the
strings, while keeping the online translation intact. Note that this
variable has no effect on the server templates
and all articles defined in the variable
Grace period for the out-of-date notice. When the variable
GRACE is defined,
OUTDATED-GRACE defaults to 60 days
(see grace period). The out-of-date notice is a special
text (server/outdated.html in the ‘www’ repository) that is
inserted into every outdated translation when the period defined in
this variable is over; its purpose is to inform the reader that the
translation may not correspond to the original English article.
The translation team whose articles need to be checked for
completeness. This variable is applicable only for the
target, and is mandatory for it. See report.
When validation is enabled, the
original English articles are validated first, before any commands
that generate the other files, and
make exits with an error
on the first encountered article. This is done on purpose, to prevent
the propagation of an eventual error in the markup of the original
article to all translations.
Validation of the translated .lang.html is performed after it is preliminarily generated as a temporary file. When no errors are found, the translation is updated; otherwise the real file is not changed (and it is not added if absent)—the build will fail and further processing of the remaining articles will not be performed. The translator has time until the next run to fix the error—usually by modifying the corresponding .lang.po file.
If notification is enabled (
NOTIFY=yes), and the build system
encounters errors (mostly when validating articles), email messages
will be sent to the party that is expected to fix the error. The
subject of the messages always includes the problematic article, for
Subject: [GNUN Error] gnu/gnu.fa.html is not valid HTML
Some targets are not built by default, because they are only useful
under certain circumstances. Think of them like semi-automated
commands or canned command sequences that are more complicated, and
more importantly, whose arguments are variables computed at the time
make reads the makefiles—the filesets they affect are
specific and already defined, one way or another.
no-grace-items target regenerates a limited set of articles
that are not affected by the grace period, namely, the
server templates, and all articles defined in the variable
no-grace-articles (see grace period).
This target can be rebuilt more often than
currently we just build all with the same period.
update-localized-URLs target invokes a script that
generates the localized-urls.mk file. This file includes the
list of URLs that should be localized, it is extracted from special
comments in the HTML source (see Localized URLs).
This target is meant to be rebuilt nightly.
This target exists solely for convenience to translators, enabling them
to check which articles are not 100% translated and have to be updated.
The way to check this is by running
make report TEAM=lang,
where lang is the language code, as usual. Thus, to check all
French translations, one would run
make report TEAM=fr
This target checks only the PO files; the old translations that haven’t been converted to PO files are reported, but there is no reasonable way to check if they are up-to-date. In fact, this is one of the main reasons GNUN is being developed, if you recall.
When present, priorities.mk defines four classes of articles by priority: priority-articles for the most important translations, important-articles for the second priority level, important-directories for the directories with important articles; all other translations are reported as less important.
There is also a script,
gnun-report, to generate HTML reports.
See gnun-report. GNUmakefile.team provides another
target, which produces a more detailed report.
See report in GNUmakefile.team.
This is a special target intended to be run by the automatic build after
the main build and after
triggers target currently executes the files named
article.lang.html.hook in the server/gnun
directory—these files are created during the main build and each of
them contains the command to update the timestamp of the prerequisite
based on the timestamp of the target that must be rebuilt. Finally, it
deletes all those *.hook files.
This is the rule that takes care of actually sending the
ANNOUNCE=yes. Since it is a completely valid
scenario to have a new translation which is initially invalid HTML,
the canned command sequence for announcements is recorded in
*.hook-ann files, and they are treated by
triggers in a
different way. A newly added .lang.po file may be
invalid, in which case the HTML file is not added, and it is not
appropriate to announce it as a new translation as it is completely
useless for the
general public. The
triggers rule detects this case, and
postpones the announcement to the next build attempt, when the
generated HTML translation is supposed to be human-readable.
To summarize, for effective operation GNUN should be invoked
make; cvs|svn|bzr commit -m …; make
triggers. To illustrate this, here is a concrete example showing the
official job once running at fencepost.gnu.org5:
25 4,12,20 * * * cd $HOME/projects/www; cvs -q update &>/dev/null; \ make -j1 -C server/gnun VCS=yes VALIDATE=yes; \ NOTIFY=yes VERBOSE=yes GRACE=5000; ANNOUNCE=yes; \ cvs commit -m \ "Automatic update by GNUnited Nations."; \ make -C server/gnun triggers
The above example is for CVS; if the underlying repository is
Subversion, they need to be amended accordingly. If it is GNU Bzr,
remember to add a
bzr push after commit (in the usual
scenario), otherwise changes will be committed only locally. Since a
distributed Version Control System can be used in multiple (sometimes
radically different) ways, this step cannot be anticipated and therefore
cannot be automated. Adding the
push command in the makefile
rules would not work if a so called “bound branch” is used, for
In the future, this target may be extended further to do other useful things that should be “triggered” after the main build.
validate-all target validates all HTML pages under GNUN’s
control. It is needed because GNUN doesn’t track dependencies on the
included files, so errors in those included files could pass unnoticed.
This target is invoked once a day by the official GNUN cron job.
The file gnun.mk contains variable definitions, based on which almost all other important variables are computed. In other words, the variables defined in that file directly affect the overall behavior of the build process.
There are two types of variables, which are specifically separated in
order to make translators’ life easier: variables that translators are
free to modify and variables that are modified by the
web-translators6, ideally after performing some local tests. A
translation team leader should update only
TEMPLATE_LINGUAS; everything else is supposed to be built
automagically, without manual intervention. If not, that is a bug that
should be reported and fixed.
You can also request building files separately this way:
make -C server/gnun ../../philosophy/not-ipr.bg.html
A space-separated list with languages. Add here your language code
if and only if you have all the SSI templates translated, and
have already committed all template files listed in
Add your language code here if you want GNUN to add differences to
msgids in your PO files.
See gnun-add-fuzzy-diff, for more information.
This variable lists the templates that are not under GNUN’s control and are translated manually, like
They contain HTML code and SSI directives; PO4A doesn’t extract them to any messages in PO files.
extra-templates variable lists templates under GNUN
control; they are rebuilt from corresponding
when the template.lang.po file is absent, GNUN initializes
and commits a file with empty
optional-templates variable defines optional templates under
GNUN control. Those are the templates of low priority items, like news
lines included in some pages. They are managed like the additional
templates listed in the extra-templates variable, except
This way, the scripts reporting outdated translations and translations that haven’t been converted to PO files won’t complain about them unless the team decides to actually commit template.lang.po.
sitemap variable declares the pages that are treated like
sitemaps, that is, an additional externally generated compendium is used
when updating them. See Sitemap, for more information.
Add here articles that are in the server root, like home.html
and keepingup.html. Always write only the
basename of the article, i.e. if you add these two articles, the value
ROOT should be
home keepingup. This is true for
all the variables that expect values in the form of article names.
The list of directories containing articles, like philosophy, gnu, licenses, etc.
A space-separated list of basenames for articles residing in directory, for which POTs will be generated and updated when the original article changes. If an article is missing here, GNUN doesn’t maintain its translations.
The file server/gnun/languages.txt is used when generating lists of translations; those lists are subsequently included in all translations of the article. Every line in the file is either a comment (when it begins with “#”) or a definition for a language. A language is defined by three TAB-separated fields. The first field defines the language suffix; it is used in file names and in texts of the links. The second field is the name of the language in English. The third field defines the name of the language in that language itself; it is used in texts of the links, and it should be in UTF-8. For example:
de German Deutsch
The generated list of translations may look like this:
<!-- begin translist file --> <div id="translations"> <p> <span class="original"><a href="/distros/screenshot.en.html">English</a> [en]</span> <span><a href="/distros/screenshot.de.html">Deutsch</a> [de]</span> <span><a href="/distros/screenshot.pl.html">polski</a> [pl]</span> </p> </div> <!-- id="translations" --> <!-- end translist file -->
Compendium is a PO file including translations for common strings; it is used to fill other PO files. See Using Translation Compendia in GNU gettext tools. One example of such common strings is the footer text about reporting bugs and sending inquiries: when a webmaster updates footer texts in an article, GNUN will use compendia to automatically fill the translations for the new version of the strings.
GNUN uses compendia located in the server/gnun/compendia directory of the ‘www’ web repository. There are two kinds of compendia: master.lang.po and compendium.lang.po.
The first kind, master.lang.po, can be used to simultaneously update all occurrences of the translations of a given string. Translations from this file will override the translations from article.lang.po. When master.lang.po is updated, the translations will be rebuilt. GNUN doesn’t modify this kind of compendia.
The second kind, compendium.lang.po, is updated automatically. GNUN finds strings that repeat many times in POTs of articles and collects them in compendium.pot. Then it checks all available article.lang.po files for translations of those strings and generates compendium.lang.po. This file is also used to fill missing translations, but it doesn’t override the translations from article.lang.po, and the strings coming from compendium.lang.po are always marked as “fuzzy” to prevent propagation of translations that may be wrong in a different context.
When updating compendium.pot, some strings should be excluded even though they repeat in the POT files many times—for instance, GNUN slots for translators’ notes. See Notes Slot. They are not real translations, this is why they are likely to be different for different articles. In order to avoid including them in compendia, GNUN checks a specific file, exclude.pot, and when that file contains the string, it won’t be added to compendium.pot.
Sitemaps differ from regular pages in two respects:
Also, sitemaps contain many translated messages, so it is desirable to eliminate manual work where possible.
In order to take this into account, GNUN provides the
variable in gnun.mk. All articles listed in
rebuilt using an additional compendium (when present),
Those additional compendia are supposed to be compiled outside of GNUN
by the same tool that generates the sitemap. It is the responsibility of
that tool to provide the compendia with reasonable translations whose
msgids match those generated by PO4A when processing the
English version of the sitemap.
GNUN operates on the “official” Web repository of the Savannah project ‘www’, where normally only the co-ordinators of translation teams have write access. However, all translation teams have their own projects, so it is possible to take advantage of Savannah as a hosting facility to make the team work more comfortable.
The PO files provide an excellent and natural way to review each other’s translations, because the translation appears right below the original message. Mutual reviews and proof-reading of translations is a crucial part of the process. Furthermore, team work is great for the community spirit; automating some of the operations also result in more time for all members to concentrate on the important tasks.
The file GNUmakefile.team in the ‘gnun’ package is a template, aimed for all translation teams who wish to use their own project’s repository as a place to keep their draft translations, until they ripe and are ready to be installed officially.
The following diagram illustrates a typical workflow—it is applicable for small, medium and large teams:
+----------+ +-------------------+ | ``www'' | | ``www-LANG'' | | Web |------>---->-------|Sources repository | |repository| automatic merge +-------------------+ +----------+ | | | | | | `-- Member A +------------<----<-------------' | Leader `---Member B
All members and the team leader commit in their project’s repository—when a translation is ready, the leader checks it in the official ‘www’ repository. If an original article changes, a build can be invoked to synchronize (i.e. merge) the changes and optionally automatically commit them so that the draft PO files are updated; at the same time, automatic notifications are sent to the persons interested in particular translations7. A translator will then normally update the PO file, and commit it again in the project’s Sources repository, from where the co-ordinator will pick it up and install it in ‘www’.
To take advantage of this semi-automation, rename this template GNUmakefile.team as GNUmakefile and install it in the root of your project’s Sources repository. Then create directories and sub-directories exactly as they are in ‘www’. Do not create the /po sub-directories; they are redundant here. Instead, install the PO files in the normal locations where the corresponding .lang.html resides in ‘www’, for example:
Root | |--GNUmakefile |--home.lang.po |--… |--gnu | | | | | +--linux-and-gnu.lang.po | +--manifesto.lang.po | +--… | | +--philosophy | | | | | +--free-sw.lang.po | +--not-ipr.lang.po | +--open-source-misses-the-point.lang.po | +--… | +--…
The next sections explain how to adopt the makefile for your team and how to invoke a “build”.
|• GNUmakefile.team Variables:|
|• GNUmakefile.team Targets:|
|• GNUmakefile.team and Cron:|
To adjust the makefile for your team, you need to edit the next variables.
Set this to the language code, like
It is autodetected if not defined.
The relative or absolute path to the working copy of the master ‘www’
repository. So if you have checked out your project’s Sources
repository at ~/projects/www-lang and the ‘www’ Web
repository at ~/projects/www, the value of
../www/ or /home/user/projects/www/. Note the
slash at the end, it is important.
These variables are used to transform the path to a file in the team’s working copy into a URL for this file; they are only needed for automatic email notifications. It is expected that typically the URL will point to some sort of Web interface for the repository, for example,
TEAM_URL_PREFIX := http://cvs.savannah.gnu.org/viewvc/*checkout*/www-bg/ TEAM_URL_POSTFIX := ?root=www-bg
If two variants of one language share the same project and repository
zh-tw), they should maintain two
directories with two GNUmakefiles and each directory having its
Some variables are specified on the command line, and alter the behavior of the build process.
The program to manage differences to “previous”
msgids added to
“fuzzy” translations when merging PO files. When undefined,
the makefile tries to find
gnun-add-fuzzy-diff and assigns
the result to this variable. See gnun-add-fuzzy-diff, for more
Define to empty string to disable the feature.
Default period of reminders sent by the
notify target actually send notifications.
Do not send email notifications about errors. This is the default.
Print more information from
msgmerge; off by default. Note that
VERBOSE can be
defined to any string, it will have the same effect.
Update both ‘www’ and ‘www-lang’ repositories, then commit the merged PO files in the latter repository. By default, there is no VCS interaction. The VCS of the translation project repository is determined automatically; currently only CVS, Subversion, GNU Bzr, Git, Mercurial (Hg) and GNU Arch repositories are supported.
Caution: The makefile rule will commit all local changes, not
only those that resulted from running
msgmerge. Thus, it is
better to use a separate working copy dedicated solely for this
Updates the repositories. Does nothing unless
Merges all available PO files from the corresponding POT in www. If a POT is missing in the master repository (usually, because it was deleted when the original article was either split, renamed or deleted), a warning is printed for the corresponding file and no merging occurs for it.
Verifies which translations are complete, and prints a list (with statistics) of those that need to be updated. Also, it reports the translations that are not consistent with the revisions from www repository.
More fine-grained notifications are implemented as the
report target, filters its output for different
translators and sends them notifications.
As an addition to the output of the
report target, URLs for
relevant files are written; also, when www and team revisions
are not consistent, their differences generated with
gnun-diff-po are optionally attached (see gnun-diff-po,
for more details).
The list of files to report against are configured with nottab file. The lines of this file that begin with ‘#’ are ignored; other lines should contain two fields separated by ‘:’.
The first field is an extended grep regular expression; the files are selected for the report if their names match this expression. See Regular Expressions in GNU Grep manual.
The second field is a space-separated list of single-word translators’ identifiers for which the line applies. The file may look like this:
^gnu/gnu-history: alice bob carol ^distros/(common-distros|distros|screenshot): dan alice ^(licenses|philosophy)/: eve bob frank ^licenses/: dan
Email addresses of the translators are kept in a separate file, email-aliases. Like in nottab, the lines beginning with ‘#’ are ignored; other lines contain fields separated by ‘:’.
The first field is the translator’s identifier, the second is space-separated list of translator’s email addresses. These two fields are mandatory.
The third field is the personal reminder period in days. After that
period the message is sent again even though its contents don’t
change. Otherwise, the
notify target only sends messages when
something changes. Naturally, no message is sent when no action
is needed in the requested set of translations (unless the
force option is used).
The team-wide default period is defined in its GNUmakefile.team using
The fourth field provides comma-separated options for the report. Currently recognized options are:
Forces sending messages even when there are no files to work on.
Disables sending www vs. www-lang differences as attachments.
Report cases that suggest actions by the team leader, that is, when the translation in the team repository is complete whereas in www it is incomplete or absent.
This is an example of email-aliases:
alice:firstname.lastname@example.org email@example.com # Disable `dan' temporarily. #dan:firstname.lastname@example.org:5:no-diffs,www bob:email@example.com:3 # Note that the third field is empty; the default value for period # will be used. carol:firstname.lastname@example.org email@example.com::no-diffs eve:firstname.lastname@example.org:2:www frank:email@example.com:1:www,no-diffs
The email-aliases file should be kept privately because it may contain personal email addresses of people. How to do this is out of scope of GNUN and this manual.
A convenience rule to re-wrap all files up to the standard length. Many
PO editors leave the
msgstr as a single long line after it has
been edited, but GNUN will automatically re-wrap the file to the standard
line length when it is processed. Wrapping long lines in PO files is a
good practice as it avoids unnecessary revisions.
This rule filters all translations with
it reformats the lines that are longer than the default page width
publish rule’s task is to copy all modified files to the
official www repository. It depends on the
to ensure that all files are re-wrapped to the standard line length
limit, but deliberately does not depend on
Usually, one would run
make publish when one or a bunch of PO
files are in a satisfactory condition to be published, and this rule is
just a convenience to avoid multiple manual
As a rule of thumb, before running this rule it is sane to run
sync and correct any “fuzzy” messages and other problems, if
publish rule does not depend on
because that would be a nuisance for offline operations and basically
unnecessary when the committer is fairly confident that there are no
changes to (re-)merge. A hard dependency on
sync would slow down
the operation considerably.
As usual, when committing to the official repository, it is always a
good practice to examine the output of
make publish prints warnings and does not actually copy
the affected file if the sub-directory in “www” is non-existent or the
corresponding .pot is missing.
Typically, after an editing session (whether it involves actual editing or just merging contributions from team members), one would do:
$ make sync VCS=yes $ make publish $ cd wwwdir $ cvs up (Add all new translations, if any.) $ cvs add file … $ cvs diff $ cvs commit -m "Some descriptive message."
Deletes all backup and auto-generated files.
make VCS=yes is the recommended command to be run periodically.
To check the status of the translations, run
Feel free to replace all strings with equivalents in your native language and of course—do not hesitate to extend this file and modify it as much as you like. If you come up with something interesting, it would be nice to send a message to firstname.lastname@example.org, so that GNUmakefile.team gets updated for all teams’ benefit.
It is convenient to invoke such synchronization automatically, for example once every day. If you have enabled commit notifications for the project’s repository, any new changes will be visible for subscribers. Here is an example crontab entry:
# m h dom mon dow command @daily cd $HOME/projects/www-lang; make VCS=yes
The job doesn’t have to run on the team leader’s machine, since all team members have write access to their project repository.
If desired, you could set up another job to report the status of the translations weekly or fortnightly, for example:
# m h dom mon dow command @weekly cd $HOME/projects/www-lang; \ make report | mail -s "Weekly statistics" \ email@example.com
Caution: Most cron implementations do not allow the character ‘\’ as a line continuation character—the example shown is made that way for better readability.
This section provides technical details about working with PO files. For general information about PO editors, see Working with PO Files in GNU Web Translators Manual.
|• New Translation:||How to start a new translation.|
|• PO Header:||How to fill the header of a new translation.|
|• GNUN Slots:||Special GNUN messages.|
|• Wrapping Long Lines:|
|• PO Tips:||Tips and hints for translators.|
|• Migrating:||How to migrate an existing translation to a PO format under GNUN’s control.|
To start a new translation, the most simple way is to copy the
existing POT as article.lang.po, where lang is your
language code. The name of the POT is article.pot or,
when you are translating an optional template, article.pot.opt
(see optional-templates). All these files are situated in the
po/ subdirectory of the directory containing the HTML file
of the article. See Files and Directories, for more info on GNUN
directory layout. For example, to prepare for a new translation
of the essay https://www.gnu.org/philosophy/free-sw.html
in Manx, you can simply
cd philosophy/po; cp free-sw.pot
free-sw.gv.po and then edit the latter. If
free-sw.pot does not exist it is because either the article is
not yet “templated” (i.e. migrated to the new style), or the GNUN
maintainers have not yet added it to the value of the appropriate
variable in server/gnun/gnun.mk. In that case, just ask them
to do what’s needed for the POT to be generated.
You could also use the
msginit utility that would populate
the PO file header with the right information, provided your
environment is set up correctly. See msginit Invocation in GNU gettext tools.
GNUN also provides a customized script to automatically fill more header fields. See gnun-init-po.
The PO file header as generated usually looks like this:
# SOME DESCRIPTIVE TITLE # Copyright (C) YEAR Free Software Foundation, Inc. # This file is distributed under the same license as the PACKAGE package. # FIRST AUTHOR <EMAIL@ADDRESS>, YEAR. # #, fuzzy msgid "" msgstr "" "Project-Id-Version: PACKAGE VERSION\n" "POT-Creation-Date: 2008-02-06 16:25-0500\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" "Language-Team: LANGUAGE <LL@li.org>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=CHARSET\n" "Content-Transfer-Encoding: ENCODING\n"
You have to edit the header to match the already established conventions, and the rules for gnu.org translations. For reference, here is a list with all fields explained:
Add here the filename of the original article, without the sub-directory, like “body-include-1.html” or “free-sw.html”.
Do not edit this field, it is already set when the POT is created.
Likewise, do not edit. This field is automatically filled in when you save the file with any decent PO editor.
The name and email address of the last translator who has edited the translation. Pay attention that normally this is the name of a member of your team, it can be the translation team leader if he/she was the person who updated the translation. For example:
Elvis Parsley <firstname.lastname@example.org>
This field should contain the mailing list on which the translation team can be reached—usually www-LANG-…@gnu.org. Example:
Leave it like it is.
Usually this is
text/plain; charset=UTF-8; change the charset
Set this to
8bit. Note that the PO file header ends with this
field, and it should contain a newline (‘\n’). Unfortunately, some
PO editors remove the newline, which causes an unnecessary revision when
the file is automatically modified by GNUN’s rules.
Here is an example of a properly edited header:
# Bulgarian translation of http://www.gnu.org/philosophy/free-sw.html # Copyright (C) 2008 Free Software Foundation, Inc. # This file is distributed under the same license as the gnu.org article. # Yavor Doganov <email@example.com>, 2008. # msgid "" msgstr "" "Project-Id-Version: free-sw.html\n" "POT-Creation-Date: 2008-02-06 16:25-0500\n" "PO-Revision-Date: 2008-02-09 15:23+0200\n" "Last-Translator: Yavor Doganov <firstname.lastname@example.org>\n" "Language-Team: Bulgarian <email@example.com>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n"
Notice the absence of the “fuzzy” marker; you should “unfuzzy” the header after entering the necessary information (this is done by simply pressing TAB in PO mode).
It is recommended that you wrap all lines in the comments to be less
than 80 characters; that looks better from a purely aesthetic point of view
and improves the performance of GNUmakefile.team’s
rule (see publish).
There are some special messages that appear in the POT and PO:
This is for translator’s notes that are injected in the resulting translation. See Notes Slot, for more information. If your translation does not have notes, you must translate this as a space:
msgid "*GNUN-SLOT: TRANSLATOR'S NOTES*" msgstr " "
This is again optional, and should contain the name (and address) of the person who made the translation. “Translate” this string as a space if you do not want your name to appear there. See Credits Slot.
|• Notes Slot:||How to handle translator’s notes.|
|• Credits Slot:||Translator’s credits.|
Sometimes it is necessary to complement the translation of an essay
with translator’s notes. The special message
TRANSLATOR'S NOTES* is designed to serve this purpose. If your
translation doesn’t have notes, you should “translate” the
msgstr as a space (SPC)—otherwise the PO file will be
considered incomplete, which is not what you want. Here is an example
how to use translators’ notes in a PO file:
# type: Content of: <p> msgid "" "To understand the concept, you should think of <q>free</q> " "as in <q>free speech,</q> not as in <q>free beer.</q>" msgstr "" "Translated message, where you want to clarify beer<sup><a " "href=\"#TransNote1\">1</a></sup>, presumably because the " "expression in your language is different" … … # type: Content of: <div> #. TRANSLATORS: Use space (SPC) as msgstr if you don't have notes. msgid "*GNUN-SLOT: TRANSLATOR'S NOTES*" msgstr "" "<h3>Translator's notes</h3>\n" "<ol>\n" "<li id=\"TransNote1\">Note clarifying the text.</li>\n" "</ol>\n"
Certainly, everything in the
msgstrs should be in your native
language; we use English here, so that everyone understands
the example. If you have more notes, each subsequent one should be with
incremented number, i.e. ‘TransNote2’, ‘TransNote3’, etc. and
you have to add them as more
<li> elements accordingly.
Do not worry about the
\n character—it is inserted
automatically when you press RET. It is not compulsory that
notes start on a new line, this is the recommended way simply because
it is easier to edit them.
It is important to follow this specification, because notes will look consistent in all languages and will be clearly distinguishable from authors’ footnotes, if any. Furthermore, it will be easier to define a special CSS class for them, and also to convert the translations in other formats such as Texinfo—when these conversions are implemented.
Most of the translators usually put their name under the translation, in the “footer” area. This is entirely acceptable, since some readers prefer to send buggestions directly to the translator. Also, giving credit where credit is due is a natural thing.
Like the previous slot, you should “translate” it as a space if you don’t want your name to appear there.
Here is an example of the recommended way to specify credits:
<b>Traduction</b>: Benjamin Drieu <a href="mailto:firstname.lastname@example.org"><email@example.com></a>, 2007, 2008.
It is highly desirable to use this form, but you may omit the email address or add a link to translator’s noncommercial personal home page, provided that the translation team leader ensures that it constantly meets the linking criteria for gnu.org. Please follow the FSF HTML Style Sheet when adding URIs or other information.
Most of the PO editors do not wrap long lines that inevitably appear in
msgstrs. If that happens, long lines make reading subsequent
diffs harder, and are generally annoying for most people. If this issue
bothers you, you can “normalize” the already finished PO translation
by executing on the command line
msgcat -o file.po
file.po, before installing it in the repository. Either way, the
build system will treat it as a valid PO file.
For those lucky Emacs users, here is a code snippet that you can put in your .emacs; doing M-x po-wrap while in PO mode will wrap all long lines:
(defun po-wrap () "Filter current po-mode buffer through `msgcat' tool to wrap all lines." (interactive) (if (eq major-mode 'po-mode) (let ((tmp-file (make-temp-file "po-wrap.")) (tmp-buf (generate-new-buffer "*temp*"))) (unwind-protect (progn (write-region (point-min) (point-max) tmp-file nil 1) (if (zerop (call-process "msgcat" nil tmp-buf t (shell-quote-argument tmp-file))) (let ((saved (point)) (inhibit-read-only t)) (delete-region (point-min) (point-max)) (insert-buffer tmp-buf) (goto-char (min saved (point-max)))) (with-current-buffer tmp-buf (error (buffer-string))))) (kill-buffer tmp-buf) (delete-file tmp-file)))))
It is highly desirable that you check if the PO file you finished
translating (or editing) is valid, before committing it. This is done
msgfmt -cv -o /dev/null file or by simply
pressing V in PO mode. The build system automatically verifies
each PO file when invoked with
VALIDATE=yes, but you won’t get a
warm and fuzzy feeling if a stupid typo you made halts the whole update
of all translations. Such things happen to everyone, so it is a good
practice to check before you actually commit.
This section contains additional explanations, some in the form of advice and recommendations; not all of them are strictly related to PO files editing.
In the next build, your article.lang.html will be built and the link to it will be added to the list of translations and propagate to all translations, provided that they are under GNUN’s control.
TEMPLATE_LINGUASfor you, as appropriate.
msgstrand edit the latter by translating the English text. In PO mode of Emacs, this is done by C-j. This is useful also for large chunks of text in
<pre>elements, which normally you would want to preserve verbatim.
msgstrusing M-q in Emacs (or other means) is considered harmful. It is best to leave GNUN (or more precisely, Po4a) do the wrapping—that way all generated HTML translations will have predictable results. This will help tremendously for the conversion to other formats, like Texinfo. Also, note that not all elements are wrapped by default, so deliberately wrapping the text inside the
msgstrcould lead to an invalid page or a page that is valid, but is rendered incorrectly by the web browser.
Migrating an existing translation to a PO file format is basically editing the header as described in the previous section, and populating each of the messages by copying the already translated text and markup from the existing translation in HTML format in the relevant message.
Typically, you will visit po/foo.lang.po (in PO mode) and
foo.lang.html (in HTML mode) in another buffer. Then you
can copy a paragraph or an element from the latter and yank it in the
relevant message in the former. Be extra careful, since this is the
time to check precisely that the translation corresponds to the
original. Further changes will be reflected, but if your “initial”
PO file is not a 100% match, that would not necessarily mean that it
is an improvement. Since it is very easy to do this kind of check,
because the relevant
msgstr appear one above
the other in the same buffer (or the similar concept in other PO
editors), please do perform this initial sanity check even if
you are confident that the translation you have been yanking strings
from is a completely up-to-date translation.
There is also a semi-automatic way to produce an initial PO file. You
checkout the revision of the English page, foo.html, that
corresponds to the latest revision of the translation,
foo.lang.html. Then you run
gnun-preconvert foo.lang.html foo.html
If some passages in foo.lang.html don’t match the structure
of foo.html, error messages will be displayed. Check them,
adjust the files and try again. When you succeed, the result will be
written to foo.lang.po. After that, run
gnun-merge-preconverted (see gnun-merge-preconverted):
gnun-merge-preconverted -C compendium.lang.po foo.lang.po foo.pot
If you have no compendium, just omit the “
You get foo.lang.po where all messages are marked as
“fuzzy” (unless you use a compendium); you still should make sure that
the translations correspond to the original and remove those “fuzzy”
marks. The script adds differences against “previous”
There is no need to delete the existing HTML translation, GNUN will automatically overwrite it. The only thing the translator should do is commit the PO file to the repository.
When an essay has been translated by several people through the years, it is important that this information is recorded and reflected in the PO file. In the future, special targets may be added to enable the FSF to check who translated a particular article, and when.
A recommended way to do this is as follows:
# French translation of http://www.gnu.org/philosophy/bsd.html # Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc. # This file is distributed under the same license as the gnu.org article. # Cédric Corazza <firstname.lastname@example.org>, 2006, 2008. # Jerôme Dominguez <email@example.com>, 2007.
In this example, it is clear that Cédric made the initial translation, Jerôme made some changes in 2007, and the original translator returned in 2008 and continued maintaining it.
This section contains some tips and general recommendations for webmasters in no particular order—it is not mandatory to follow them, but doing so will make translators’ lives substantially easier.
First and foremost, respect translators’ work—it is ungrateful and hard, undoubtedly much harder than translation of programs. It is important to have as many translations as possible, and make them as good as possible, and you don’t have to make titanic efforts to help.
If you plan to edit a certain page extensively, please do so within a reasonable time frame, for example a day or preferably, several hours. That way, the translators who are quick to update it immediately won’t be disappointed if the POT changes again in the next GNUN run.
|• Modifying Templates:||How to avoid errors.|
|• Comments for Translators:||Passing comments to translators.|
|• Localized URLs:||Specifying URLs of diagrams to be translated.|
|• Splitting Long Passages:||How to avoid too long strings in PO files.|
Any significant structural diversion from boilerplate.html in a specific article may result in errors from GNUN. Any untested intrusive updates to the server templates (such as changing the entire look & feel of the site) will probably break GNUN build process. Of course, this does not mean that no changes should happen—only that they must be applied in a sandbox first, to ensure a smooth transition.
gnun-validate-html is useful for webmasters who
want to verify if their (potentially intrusive) changes result in a
valid markup. Before committing your changes, you can check whether
the file is valid by running
gnun-validate-html --root . philosophy/not-ipr.html
See gnun-validate-html, for more information.
If you want a comment to be visible for translators, place it inside the element, for example:
<p> <!--TRANSLATORS: Note that foo is bar in this context.--> The fooish bar mumbles bazzling. </p>
This will result in:
# type: Content of: <p> #. TRANSLATORS: Note that foo is bar in this context. msgid "The fooish bar mumbles bazzling." msgstr ""
As per the established convention, start the comment with
TRANSLATORS: to catch their attention, and do not add a space
after the beginning of the HTML comment (
<!--), since this will
unnecessarily indent the comment in the POT.
Some articles may contain diagrams or other images with English text that can and should be translated. In order to make the translated versions appear in the respective translations, GNUN should be told what URLs need localization. It can be done with HTML comments like
<!-- GNUN: localize URL /philosophy/category.png, /licenses/template-diagram.png and /graphics/jesus-cartoon.jpg -->
The URLs are separated with spaces. One trailing comma at the end of every word is removed if present. Words without a dot, such as ‘and’, do not count as URLs; they are ignored.
Such comments will be extracted nightly and compiled into per-article lists of URLs in localized-urls.mk.
After every build GNUN will check if the respective philosophy/category.lang.png and other files are present in the working copy and substitute the strings in the HTML file of the translation.
GNUN relies on URLs being absolute, starting from the root directory as required in the FSF HTML Style Sheet Guidelines.
And please don’t forget to commit the image in its source form (typically, in SVG format).
GNUN splits typical text by paragraphs, and translators write the translations for the resulting parts of the text one by one. When a part is too long, it is hard to translate and proofread the translation; the likelihood of skipping some phrases is relatively high.
In order to make our translators’ life easier, it is desirable to keep
the paragraphs short (no more than 350–700 characters). If rearranging
the paragraphs is not an option, you can use a conventional separator,
<p>GNUN splits typical text by paragraphs, and translators write the translations for the resulting parts of the text one by one. <span class="gnun-split"></span> When a part is too long, it is hard to translate and proofread the translation; the likelihood of skipping some phrases is relatively high.</p>
It is important that the separator be inserted between complete sentences, because different languages may require different orders of parts of the sentence, and the translator has no control over the sequence of the strings in the translation.
This chapter might be of interest probably only to people who would have special interest in the software, plan to enhance it or develop a front-end, except the section about scripts that includes descriptions of programs that may also be useful for the translators.
|• Files and Directories:||What, where, when.|
|• Scripts:||Helper scripts.|
This is a brief diagram of www working copy; note that in this section we don’t describe the files out of GNUN control, including the old HTML-only translations.
Legend: * The file is maintained locally, without committing it to CVS. # The file is edited manually. @ The file is regenerated automatically. % The file is edited both manually and automatically. #/@ The file is generated automatically, but outside of GNUN.
Root/ | +--gnu/ | | | +--linux-and-gnu.html # | +--linux-and-gnu.lang.html @ | +--manifesto.html # | +--manifesto.lang.html @ | +--… | | | +--------------------------po/ | | +--home.html # +--manifesto.lang.po % +--home.lang.html @ +--manifesto.pot @ +--planetfeeds.html #/@ +--manifesto.proto * @ +--planetfeeds.lang.html @ +--manifesto.translist @ +--po/ | | | +--manifesto.lang-en.html @ | +--home.pot @ +--manifesto.lang-diff.html @ | +--home.proto * @ +--linux-and-gnu.lang.po % | +--home.translist @ +--… | +--home.lang.po % | +--home.lang-en.html @ | +--home.lang-diff.html @ | +--planetfeeds.proto * @ | +--planetfeeds.pot.opt @ | +--planetfeeds.lang.po % | +--… | +--server/ | | | +-------------------------gnun/ | | | | +--sitemap.html #/@ +--GNUmakefile * # | +--sitemap.lang.html @ +--config.mk * # | +--po/ +--gnun.mk # | | | +--priorities.mk # | | +--sitemap.pot @ +--languages.txt # | | +--sitemap.proto * @ +--compendia/ | | +--sitemap.lang.po % | | | | +--master.lang.po # | | +--body-include-2.pot @ +--compendium.lang.po @ | | +--body-include-2.proto * @ +--compendium.pot @ | | +--body-include-2.lang.po % +--excluded.pot # | | +--… | | | +--body-include-2.html # | +--body-include-2.lang.html @ | +--… +--…
A description of some files follows.
The English files.
The lang translation of …/file.html.
Intermediate files generated from file.html; they are used only internally. Essentially, GNUN inserts its special slots. See GNUN Slots, for more details.
The list of translations for …/file.html. It is included in …/file.html and all its translations.
PO templates for articles and included files.
POTs of optional templates. See optional-templates, for more details.
Translations of file.pot or file.pot.opt committed by the translators and merged by GNUN.
The latest revision of the English file for which a full translation into lang language was provided.
The highlighted differences between the current version of …file.html and …/po/file.lang-en.html.
A list of news; it is automatically generated outside of GNUN. For GNUN, it is an optional template.
Most template SSI files are located in this directory.
The sitemap; it is generated automatically by a script that is not included in GNUN. See Sitemap.
The directory for GNUN files.
These files are maintained locally. They are used to rebuild the translations. See Invoking GNUN.
The file that defines the list of files to build. See Main Variables.
This file defines the list of translation priorities. See priorities.mk.
Canonical names for languages. See languages.txt.
The directory for compendia. See Compendia.
For the time being there are several helper scripts. Some of them are used internally as commands with certain arguments in the makefile rules; other scripts were written specifically to facilitate some mechanical actions, like initially filling the headers in the PO files. They all can be invoked separately, as stand-alone programs.
|• gnun-add-fuzzy-diff:||Add differences to “previous”
|• gnun-diff-po:||Compare two revisions of a PO file.|
|• gnun-init-po:||Initialize a new translation.|
|• gnun-preconvert:||Invoke the first step in converting HTML translation to PO format.|
|• gnun-merge-preconverted:||Run the second step of the conversion.|
|• gnun-report:||Generate report in HTML format.|
|• gnun-validate-html:||Validate HTML file.|
This script adds comments with differences of current
against “previous” ones to “fuzzy” translations in a PO file. To produce
wdiff is used. (See The word
difference finder in GNU wdiff, for more information.)
This may be useful to figure out what has changed. In fact, it wraps
sed script used in GNUN internally.
gnun-add-fuzzy-diff [OPTION...] [FILE]
Edit the file in place.
Display copyright and version information and exit.
Display usage information and exit.
The script is invoked automatically when requested through the
FUZZY_DIFF_LINGUAS variable in GNUmakefile
(see FUZZY_DIFF_LINGUAS) and through the
variable in GNUmakefile.team (see ADD_FUZZY_DIFF).
This script compares two versions of a PO file. It produces a HTML page with a table that contains original strings with highlighted differences of their translations.
If the sets of original strings in the input PO files differ, one of them will be merged with the other in order to eliminate the differences in the original strings.
The results are written to standard output. Example:
gnun-diff-po [OPTION...] PO_FILE1 PO_FILE2 > diff.html
Adjust the original strings of PO_FILE2 against PO_FILE1 when their sets differ. This is the default.
Adjust the original strings of PO_FILE1 against PO_FILE2 when their sets differ.
Suppress strings that don’t differ.
Specify the title of the output HTML page.
Display copyright and version information and exit.
Display usage information and exit.
This script initializes a PO file using the POT generated with GNUN, and fills some fields in the header. It also optionally uses a compendium (or compendia) to fill translations.
gnun-init-po [OPTION...] POT
Specify a compendium to use. You can issue this option
many times to use multiple compendia simultaneously. The suffix
of compendium is used when the language suffix is not specified
Don’t add diffs to “previous” messages.
Specify team’s name and mailing list.
Specify language suffix, e.g “bg”. The suffix also defines the name of the language which is used in some fields of PO file header.
Display copyright and version information and exit.
Display usage information and exit.
The PO file name is guessed from the name of POT and language suffix; the file is created in the current working directory.
This script uses
po4a-gettextize to convert a translation
from HTML to PO format (see
the conversion is successful, you can merge the result with the new
gnun-preconvert [OPTION...] TRANSLATION MASTER
Specify the encoding of TRANSLATION (if other than UTF-8).
Display copyright and version information and exit.
Display usage information and exit.
This script takes
po4a-gettextize output, adds current
msgids as “previous” values, merges the file with the new
POT, and adds differences against “previous” values like
gnun-merge-preconverted [OPTION...] PO POT
Specify the compendium (if any). This option can be used more than once to specify multiple compendia.
Don’t add diffs to previous messages in the format of gnun-add-fuzzy-diff.
Display copyright and version information and exit.
Display usage information and exit.
This script generates HTML reports about translations of a given team. The HTML contains a set of tables sorted by priority and translation status (existing translations that need maintenance, untranslated files, complete translations).
The script depends on the presence of the priorities.mk file in the server/gnun sub-directory of the working copy of ‘www’ repository.
The results are written to standard output. Example:
gnun-report --root=../www -t bg > report-bg.html
Specify the language code of the team, for example, ‘ml’.
Specify the top directory of the working copy; the default is the current directory.
Specify the language name, for example, ‘Malayalam’. When this option
gnun-report tries to figure out the language name
based on the language code provided with the “-t” option.
Display copyright and version information and exit.
Display usage information and exit.
A cron job commits updated reports for all active teams to GNUN project web repository, typically twice an hour. The links to those reports are provided on the GNUN Reports.
There is also a target in GNUmakefile to generate text reports intended for monthly messages sent to the teams. See report.
This is a Bash script whose purpose is to “validate” both the original and translated articles to make sure that they conform to the respective W3C standard. Sometimes webmasters make mistakes, and translators too, so this tool is useful to catch errors of that kind.
GNUN enforces HTML validation at build time by default.
The script expects only one file as the last argument and will exit with an error if it is not specified (which might be the case when an automatic variable is not expanded properly due to a bug in the makefile). Example:
gnun-validate-html --root . philosophy/free-sw.html
Specify the top directory of the working copy; the default value is ../...
Save the expanded HTML as file.
Specify additional (or override assumed) emulated predefined Apache variables.
Produce more detailed output intended for automatic email reports; essentially, it adds the expanded HTML to facilitate finding errors by people who receive the report.
Display copyright and version information and exit.
Display usage information and exit.
This is a helper script that runs a command, and mails the output of
that command in case it exits with a non-zero exit status.
mailfail depends on GNU Mailutils, or a compatible
implementation, such as BSD’s mailx.
mailfail [--dry-run] RCPT SUBJECT CMD [ARG ...]
mailfail script accepts the following options:
Does not send the email message.
The recipient of the message in a valid format, like
The subject of the message; if it is longer than a word you should guard it with quotes.
The command you want to run and send a mail in case it fails.
The arguments of
CMD, if any.
Here is a typical example, similar to the way it is used in GNUN:
mailfail firstname.lastname@example.org "Bad PO" msgfmt -cv -o /dev/null bg.po
This will check the validity of bg.po with the
program and in case there are errors, a message will be sent to the
specified address with ‘Bad PO’ as subject and the error output from
msgfmt as body.
mailfail inherits the exit status of the command being run.
If an argument is missing, the usage information is printed to the
standard output and the exit code is 1.
This script is a wrapper around
(see gnun-validate-html); it is necessary because it is hard to
capture the output of the program from a program that itself captures
the output of another program that it runs.
validate-html-notify [--dry-run] RCPT FILE
Does not actually send the message, just like
In this case it runs
--verbose because it is expected that the expanded file
will be available locally.
The recipient of the message.
The HTML file that has to be validated for compliance with the W3C standard.
The subject of the message is hardcoded in the script, since this
wrapper has a specific task and cannot be used to invoke an arbitrary
mailfail for that. See mailfail.
This simple script is not used internally in GNUN. It is merely for convenience only, for those who find it hard to remember the various ‘gettext’ tools and their options.
gnun-clear-previous deletes the “previous” messages in a
PO file, which is a good thing to do after the corresponding translation
is updated and the “fuzzy” marker removed. It can also be used to
wrap long lines in PO files.
Print copyright and version information on the standard output.
Print usage information on stdout.
GNUnited Nations, like any other software, is not bug free. There are some known bugs and annoyances, which are listed in the TODO file, but it is absolutely certain that there are more which we know nothing about.
If you encounter a bug, or if you have suggestions of any kind, please do not hesitate to report them at email@example.com or GNUN Bug Tracker.
Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. http://fsf.org/ Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
The purpose of this License is to make a manual, textbook, or other functional and useful document free in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.
This License is a kind of “copyleft”, which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.
We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.
This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The “Document”, below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as “you”. You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.
A “Modified Version” of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.
A “Secondary Section” is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document’s overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.
The “Invariant Sections” are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.
The “Cover Texts” are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.
A “Transparent” copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not “Transparent” is called “Opaque”.
Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.
The “Title Page” means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, “Title Page” means the text near the most prominent appearance of the work’s title, preceding the beginning of the body of the text.
The “publisher” means any person or entity that distributes copies of the Document to the public.
A section “Entitled XYZ” means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as “Acknowledgements”, “Dedications”, “Endorsements”, or “History”.) To “Preserve the Title” of such a section when you modify the Document means that it remains a section “Entitled XYZ” according to this definition.
The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.
You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.
You may also lend copies, under the same conditions stated above, and you may publicly display copies.
If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document’s license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.
If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.
If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.
It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.
You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:
If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version’s license notice. These titles must be distinct from any other section titles.
You may add a section Entitled “Endorsements”, provided it contains nothing but endorsements of your Modified Version by various parties—for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.
You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.
The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.
You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.
The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.
In the combination, you must combine any sections Entitled “History” in the various original documents, forming one section Entitled “History”; likewise combine any sections Entitled “Acknowledgements”, and any sections Entitled “Dedications”. You must delete all sections Entitled “Endorsements.”
You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.
You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.
A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an “aggregate” if the copyright resulting from the compilation is not used to limit the legal rights of the compilation’s users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.
If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document’s Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.
Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.
If a section in the Document is Entitled “Acknowledgements”, “Dedications”, or “History”, the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.
You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute it is void, and will automatically terminate your rights under this License.
However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it.
The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.
Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License “or any later version” applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used, that proxy’s public statement of acceptance of a version permanently authorizes you to choose that version for the Document.
“Massive Multiauthor Collaboration Site” (or “MMC Site”) means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such a server. A “Massive Multiauthor Collaboration” (or “MMC”) contained in the site means any set of copyrightable works thus published on the MMC site.
“CC-BY-SA” means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-profit corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization.
“Incorporate” means to publish or republish a Document, in whole or in part, as part of another Document.
An MMC is “eligible for relicensing” if it is licensed under this License, and if all works that were first published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008.
The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing.
To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:
Copyright (C) year your name. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License''.
If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the “with…Texts.” line with this:
with the Invariant Sections being list their titles, with the Front-Cover Texts being list, and with the Back-Cover Texts being list.
If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation.
If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.
A B C D E F G I L M N O P R S T V W
|articles in root directory, defining:||Main Variables|
|comments for translators:||Comments for Translators|
|conventional separator for strings:||Splitting Long Passages|
|conversion of existing translations:||Migrating|
|conversion of existing translations:||gnun-preconvert|
|conversion of existing translations:||gnun-merge-preconverted|
|credits, translators:||GNUN Slots|
|credits, translators:||Credits Slot|
|cron, team maintenance:||GNUmakefile.team and Cron|
|deferred generation of articles:||Runtime Variables|
|defining articles in the root dir:||Main Variables|
|defining directories:||Main Variables|
|defining templates:||Main Variables|
|directories, defining:||Main Variables|
|fuzzy strings:||Runtime Variables|
|GNUN directory layout:||Files and Directories|
|grace period:||Runtime Variables|
|initializing PO files:||New Translation|
|localized SSIs:||Main Variables|
|localized URLs:||Localized URLs|
|long lines, wrap:||Wrapping Long Lines|
|long passages, avoiding:||Splitting Long Passages|
|mail, notifications:||Runtime Variables|
|mail, notifications:||GNUmakefile.team Variables|
|new translation:||New Translation|
|new translations, notifications/announcements:||Runtime Variables|
|notes, translators:||GNUN Slots|
|notes, translators:||Notes Slot|
|notification period:||GNUmakefile.team Targets|
|notification period, default:||GNUmakefile.team Variables|
|outdated translations, notifications:||GNUmakefile.team Targets|
|output, detailed:||Runtime Variables|
|PO files, header:||PO Header|
|PO headers:||PO Header|
|POT generation, articles:||Main Variables|
|previous, diff:||Main Variables|
|previous, manipulating PO files:||gnun-clear-previous|
|project repository:||Team's Repository|
|recommendations, PO files:||PO Tips|
|reminder period:||GNUmakefile.team Targets|
|reminder period, default:||GNUmakefile.team Variables|
|repository, translation project:||Team's Repository|
|sanity checks:||Runtime Variables|
|SSIs, localized:||Main Variables|
|starting translation:||New Translation|
|team maintenance:||Team's Repository|
|team maintenance, cron:||GNUmakefile.team and Cron|
|team workflow:||Team's Repository|
|templates, additional:||Main Variables|
|templates, defining:||Main Variables|
|templates, optional:||Main Variables|
|tips, translators:||PO Tips|
|tips, webmasters:||Webmaster Tips|
|translation, new:||New Translation|
|translators’ credits:||GNUN Slots|
|translators’ credits:||Credits Slot|
|translators’ notes:||GNUN Slots|
|translators’ notes:||Notes Slot|
|triggering, build:||Invoking GNUN|
|validation, HTML, XHTML:||gnun-validate-html|
|variable, behavior:||Runtime Variables|
|variable, team:||Runtime Variables|
|webmaster tips:||Webmaster Tips|
|wrapping long lines:||Wrapping Long Lines|
A B C D E F G I L M N O P R S T V W
Actually, it is a lot closer to localization of software documentation, where typically strings (also known as “messages” in gettext’s context) are longer than strings in programs. Nevertheless, all points raised still apply.
The process of converting HTML to PO and the other way around is performed using po4a (“PO for anything”), see PO4A website.
A special group of templates, so called optional templates, have names like article.pot.opt. The PO file for them is still article.lang.po. See optional-templates.
When GNU Bzr is used, files
are added locally only; you need to take care to use
manually (or via cron) to take care of effectively adding them to the
public repository. See triggers, for a short explanation.
These days we run a more complicated script instead of this cron job in order to address some exceptional situations like concurrent builds.
Only because presumably, they are more familiar with GNUnited Nations’ internals. From a purely technical point of view, there is no difference.
The following sections describe in more details how to setup these automatic actions; however, if you have any difficulties with that, please ask the GNU Web Translation Managers to set them up for your team on our server.