The life cycle of a translation resource

One of the core features Transifex provides is handling files with translatable content (resources) in various localization formats, like XML or PO files.

Part of that functionality is to be able to import such files to its internal storage and export them, whenever the user requests them, either to ship them with his software or to translate the file to another language with his local computer. Although both operations might use some customized code to handle certain formats (especially for importing resources), there are specific steps that are followed in each case.

Importing a file

Whenever you upload a file with strings in it, Transifex will try to parse it, extract the necessary information and then store that information in the database.

Parsing

Since each format is different, there are specialized parsers for each
one. In some cases, Transifex uses a third-party parser, like
polib for PO files. In other
cases, we have developed custom parsers.

Extracting the information

The main responsibility of a parser is to extract the necessary
information from the imported file.

In case the file is the source file (that is, it is the file with
the strings in the source language), we are interested in three
things:

• The keys for the translatable strings (like the msgid entries in
  a PO file). The keys are used to uniquely match the strings in the
  source language with those in translations. We also generate a
  unique hash for each key as an identifier.
• The translatable strings in the source language, if there are any
  (like the msgstr entries in PO file). These are the actual strings
  of the source language.
• The template of the file. The template is a skeleton of the source
  file: it is mostly the same, except that the translatable
  strings have been replaced with the hashes of the corresponding
  keys, acting as placeholders. This is necessary for the export
  operation.

In case the file is a translation of the resource in a language, we
are only interested in the translations (this means that any changes
in the file are ignored).

Storing

As soon as we have the necessary information from the previous step,
we store it in the database as source entities, translations and
templates.

Exporting a file

Whenever a user asks to download a translation file in a particular
language, the file has to be exported from the database.

The procedure is quite standard for all formats. After fetching the template and the
translation strings in the requested language, we do a
search-&-replace in the template, replacing the hashes in it with the
actual strings that correspond to each hash.
Next, any format-specific operations are performed (like adding the
translator copyrights in PO files) and the result is delivered to the
user.

You can find more details for the storage engine of Transifex in the
docs.