= Python, wxPython and internationalization (i18n) =
The goal of this document is to describe how to internationalize a Python application that uses wxPython for the User Interface. It uses the attached [[attachment:mki18n.py]] script.
'''Contents'''
<<TableOfContents>>
= Background =
== I18N under Python and wxPython ==
wxPython and Python support the gettext system for I18N (internationalization<<FootNote(The word internationalization is often abbreviated as I18N: take the first letter of the word (i) followed by the number of letters in the word (18) and terminate it with the last letter of the word (n))>>).
== GNU gettext system ==
Internationalization of software is supported by the GNU [[http://www.iro.umontreal.ca/contrib/po/HTML/index.html|Translation Project]]. Although the project focuses on the goal of translating software user interfaces in as many natural languages as possible and does not impose a tool set, it recommends the use of gettext (and you should use gettext with Python and wxPython).
=== Idea behind gettext ===
The general idea behind gettext is that you write your source code in English and all the natural language strings are also written in English. The strings are inserted inside the source code, you do not use string resource identifiers. But all strings that must be translatable are surrounded a small macro call: _(). The gettext system supports a large set of programming languages including C, C++ and Python.
In C or C++ _() is a special macro that calls on gettext function macro is replaced by the preprocessor (in C or C++) by a call to a gettext function that will search for the proper string at run time using the original English string as the key to the language dictionary that is currently active. If the active language is English, no translation is performed, if the active language is something else the translation is performed if a matching string is found. In Python the _() function is either mapped explicitly to gettext.gettext() by your application or installed by the gettext class API.
The dictionaries are compiled files (files with the .mo extension<<FootNote(MO stands for Machine Object file.)>>). Now, to create the .mo files, you first parse all of your source code with the gettext tools and they generate a .pot file (.POT stands for Portable Object Template) which is a simply formatted text file that contains all of the English strings that must be translated. Each English string acts as the message identifier for that string. Below each English string is a spot for the translated version of that string. You copy the .pot file into a .po <<FootNote(PO stands for Portable Object file.)>> and give it to a human translator. Then you compile the translated .po file into a .mo file that you place inside one of the LC_MESSAGE directories of your system. These directories are named after the natural language they refer to. The natural languages supported are the [[http://www.gnu.org/software/gettext/manual/gettext.html#Language-Codes|ISO 639 language codes]]. These codes are a set of two-character language codes.
Applications normally place the various .mo files inside language target specific sub-directories of the directory ./local . The following directory tree show a directory tree for English (en), French (fr) and Spanish (es) would look like.
{{{
./locale/en/LC_MESSAGES
./locale/fr/LC_MESSAGES
./locale/es/LC_MESSAGES
}}}
The .mo files for each language is stored inside the LC_MESSAGES sub-directory under the language code directory. For example, the .mo dictionary file is located inside ./locale/es/LC_MESSAGES.
=== gettext tools ===
The GNU gettext tools are available for all OS supported by GNU. The following tools are console tools for the Win32 platform. The GNU tool package include the console programs listed in the following table.
||Program ||Description ||
||gettext.exe ||Displays native language translation of a textual message. ||
||iconv.exe ||Decodes text from one encoding to another (e.g. KOI8-R to CP1251) ||
||msgattrib.exe ||Filters the messages of a translation catalog according to their attributes, and manipulates the attributes. ||
||msgcat.exe ||Concatenates and merges the specified PO files. ||
||msgcmp.exe ||Compare two Uniforum style .po files to check that both contain the same set of msgid strings. ||
||msgcomm.exe ||Find messages which are common to two or more of the specified PO files. ||
||msgconv.exe ||Converts a translation catalog to a different character encoding. ||
||msgen.exe ||Creates an English translation catalog. The input file is the last created English PO file, or a PO Template file (generally created by xgettext). Untranslated entries are assigned a translation that is identical to the msgid. ||
||msgexec.exe ||Applies a command to all translations of a translation catalog. The COMMAND can be any program that reads a translation from standard input. It is invoked once for each translation. Its output becomes msgexec's output. msgexec's return code is the maximum return code across all invocations. ||
||msgfilter.exe ||Applies a filter to all translations of a translation catalog. ||
||msgfmt.exe ||Generate binary message catalog from textual translation description. ||
||msggrep.exe ||Extracts all messages of a translation catalog that match a given pattern or belong to some given source files. ||
||msginit.exe ||Creates a new PO file, initializing the meta information with values from the user's environment. ||
||msgmerge.exe ||Merges two Uniforum style .po files together. ||
||msgunfmt.exe ||Convert binary message catalog to Uniforum style .po file. msguniq.exe Unifies duplicate translations in a translation catalog. ||
||ngettext.exe ||Display native language translation of a textual message whose grammatical form depends on a number. ||
||xgettext.exe ||Extract translatable strings from given input files. ||
==== How to get gettext tools for Win32 ====
To install the GNU gettext on your Win32 system, follow the instructions:
* download the following files:
* [[ftp://ftp.gnu.org/pub/gnu/gettext/gettext-runtime-0.13.1.bin.woe32.zip|gettext-runtime-0.13.1.bin.woe32.zip]]
* [[ftp://ftp.gnu.org/pub/gnu/gettext/gettext-tools-0.13.1.bin.woe32.zip|gettext-tools-0.13.1.bin.woe32.zip]]
* [[http://mirrors.ibiblio.org/pub/mirrors/gnu/ftp/pub/gnu/libiconv/libiconv-1.9.1.bin.woe32.zip|libiconv-1.9.1.bin.woe32.zip]]
* Create a directory called \gnu in one of your system drive (let's say in D:)
* Extract the 3 ZIP files inside the \gnu directory. This way the executable files of the 3 packages will be stored inside \gnu\bin.
* Add \gnu\bin to your PATH.
The files listed above were taken from the following sites:
* [[ftp://ftp.gnu.org/gnu/gettext/|GNU FTP site for gettetx]] where several versions (0.10.40, 0.11.2, 0.11.5 and 0.12.1) are available.
* The [[http://www.ibiblio.org/pub/gnu/libiconv/|GNU libiconv ftp site]]. This is the ftp site for the [[http://www.gnu.org/software/libiconv/|GNU libiconv]] library. The iconv library is required by gettext. Get version 1.9.1 or later.
There are other packages maintained by other individuals. I recommend you use the one above. However, the following sites helped me getting started.
* [[http://gettext.sourceforge.net/|SourceForge page of gettext for Win32]]
* [[http://home.a-city.de/franco.bez/gettext/gettext_win32_en.html|GNU gettext for WIN32]] is a little distribution of the GNU gettext for Win32
Some cautionary notes:
You should never use a version of gettext older than 0.10.39 (because it produces .po files that cannot be used by the Translation Project without human editing). Version 0.11 is considered stable according to the [[http://www.iro.umontreal.ca/contrib/po/HTML/index.html|Translation Project]] gettext is a set of command line tools and code libraries that have been developed under the [[http://www.gnu.org|GNU]] umbrella.
=== gettext file formats ===
To be written.
== Python gettext ==
The [[http://www.python.org/doc/current/lib/module-gettext.html|Python gettext module]] provides internationalization (I18N) and localization (L10N) services for Python modules and applications. It is based on the GNU gettext system.
= How to create gettext catalog files =
To help creating the gettext binary catalog files, I wrote a Python console program called [[attachment:mki18n.py]] tha uses the GNU gettext utilities to parse Python source code and create the .po and .mo files. The [[attachment:mki18n.py]] is used to perform several tasks:
* parse all Python source code files of an application and create the .pot file for each target language.
* parse all Python source code files of an application and merge the generated .pot with existing .po (files that already contain text translated by human translator).
* create the binary .mo files from the .po files.
The [[attachment:mki18n.py]] module can also be used as an imported module inside other Python programs.
= How to write an internationalized wxPython application =
The application must contain the following code:
* the main module must import the gettext module.
* the global code of the main module must call gettext.install() to install the function _() inside the application dictionary. This call identifies the translation domain<<FootNote(The translation domain corresponds to the .mo dictionary file that is searched by the _() gettext translation function. It is selected by gettext.install(). )>> (which normally is the application name), the location of the LC_MESSAGES directories and whether Unicode is used. The following is an example of the call where the dictionary files are under ./locale/xx/LC_MESSAGES (with xx being the language code) and ivcm being the name of the application (and the domain, and the name of the .mo file):
{{{#!python
gettext.install('ivcm', './locale', unicode=False)
}}}
* For each supported presentation language, the program must create a gettext. Translation instance by calling gettext.translation(). The arguments to the translation function are the application domain (the name of the .mo file which is often the name of the application), the directory parent of the languages LC_MESSAGES directories and the list of language. For example, if your application is called ivcm and the files are stored under a directory called locales, the support for English, French and Spanish would be set up by doing the following calls:
{{{#!python
self.presLan_en = gettext.translation("ivcm", "./locale", languages=['en'])
self.presLan_fr = gettext.translation("ivcm", "./locale", languages=['fr'])
self.presLan_es = gettext.translation("ivcm", "./locale", languages=['es'])
}}}
* To activate a the translation of your application for a specified language, the language must be activated by calling the Translation.install() method. For example, to activate the French presentation, you would call:
{{{#!python
self.presLan_fr.install()
}}}
* Set the wxWindows locale by calling wxLocale() and passing the wxLANGUAGE_XX code corresponding to the selected language. The following code snippet shows you how:
{{{#!python
self.locale = wx.Locale(wxLANGUAGE_FRENCH)
locale.setlocale(locale.LC_ALL, 'FR')
}}}
* In your code, all strings must be enclosed in a _() call like this:
{{{#!python
aTitle = _("Testing internationalization")
}}}
The gettext.install(domain, localedir, unicode) call instructs the gettext system to look for the dictionary file name built from the components:
. localedir/language/LC_MESSAGES/domain.mo
= Python and wxPython modules, classes and functions =
* Python gettext
* wxPython {{{wx.Locale}}} class
* wxPython {{{wx.GetTranslation()}}}
* wxWidgets {{{wxLocale}}}
= Controlling the presentation language from the environment variables =
The following environment variables control the selection of the translation language. The system uses the language code found in the first environment variable found from the following list:
* LANGUAGE
* LC_ALL
* LC_MESSAGES
* LANG
= Tools to manage translation dictionary files =
I normally use the CRiSP editor to edit .po files and compare several versions of the .po files. There are, however, specialized tools that simplify managing gettext catalog files. These tools are listed here.
* [[poEdit]] is a cross-platform gettext catalogs (.po files) editor.
= Creating .mo files with the [[attachment:mki18n.py]] script =
The [[attachment:mki18n.py]] script helps you create .po and .mo files from your source code files for an application. I describe the process of internationalizing the ivcm application here.
All strings that must be internationalized inside ivcm.py (and its companion files) have the form _("Hello"). All strings inside the source code are normally written in English as this is the convention used by the gettext system.
To use my [[attachment:mki18n.py]] script, I write a file called app.fil that contains the names of all files inside the application (one file per line, with full or relative path. For example:
{{{
images.py
ivcm.py
ivcm_about.py
ivcm_ie.py
ivcm_usermanual.py
ivcm_wxFrame1.py
../ptz.py
../action.py
../utprint.py
}}}
Then I run mki18n -p from the directory where ivcm.py is located to parse all source files and create a 'messages.pot' file. The .pot is the original template. You keep this file untouched. If I want to support French then I copy the messages.pot into a .po file named after the domain name (in this case the application name: 'ivcm') and the target language code (in this case: 'fr'). So for French I use the file name: ivcm_fr.po. If I need to support Spanish, I copy messages.pot into ivcm_es.po and so on.
The following lines show a couple of entries inside the non translated ivcm_fr.po:
{{{
#: ivcm.py:168
#, python-format
msgid ""
"\n"
" ERROR: %s"
msgstr ""
#: ivcm_wxFrame1.py:638 ivcm_wxFrame1.py:1742
msgid "&About..."
msgstr ""
}}}
The next step is to perform the translation. You can use a normal editor to append the French string inside the ivcm_fr.po or use poEdit or any other .po editor. The result of the translation would look like:
{{{
#: ivcm.py:168
#, python-format
msgid ""
"\n"
" ERROR: %s"
msgstr ""
"\n"
" ERREUR: %s"
#: ivcm_wxFrame1.py:638 ivcm_wxFrame1.py:1742
msgid "&About..."
msgstr "&A propos de iVCM..."
}}}
Note that every line with a '#' in the first column is a comment or flag. In the example above the python-format flag shows that the strings were extracted from Python source. The #: lines show the line number of the original source.
Some of the flags are set when you re-synchronyze the translations with the source. This resynchronization is required if the source changes after you have created the translated .po file(s).
My [[attachment:mki18n.py]] script will automatically perform syncronisation if it finds .po files that have the domain_language.po name layout. After a re-synchronization, 'mki18n -p' creates a .new file for every .po file found. In my example, it would create a ivcm_fr.po.new and a ivcm_es.po.new
If the source has not changed, the .new files are equal to the .po file. Otherwise, the .new file contains the new strings to translate, place the string that were removed from the source as comments inside the .po.new file and may also flag some strings as 'fuzzy'. A fuzzy flag indicates that the translation of the original source should probably change because the original string changed. So, I compare the .po and .po.new, and edit whatever is requiered, leaving the finished work inside the .po file.
The final step is to compile the finished .po file into the .mo file.
The .mo file normally reside inside the LC_MESSAGES of a 'locale' sub-directory with a xx/LC_MESSAGES for each supported language:
{{{
./locale
./locale/en
./locale/en/LC_MESSAGES/ivcm.mo
./locale/es
./locale/es/LC_MESSAGES/ivcm.mo
./locale/es/LC_MESSAGES/wxstd.mo
./locale/fr
./locale/fr/LC_MESSAGES/ivcm.mo
./locale/fr/LC_MESSAGES/wxstd.mo
}}}
The ivcm.mo in the fr and sp sub-directories were created mo file by compiling the ivcm_fr.po and ivcm_es.po by using the single command mki18n -m. The ivcm.mo stored inside the en sub-directory was created using the mki18 -e command. The mki18n is a Python scrip that uses the GNU gettext utilities.
The wxstd.mo files contain the compiled string dictionaries for wxPython. These files are distributed with wxPython and are stored under the Python/Lib/site-packages/wxPython/locale/xx/LC_MESSAGES directories (where xx is the language code). Just take them can copy them in your locale directories.
When your application runs, it uses the .mo files identified by your domain (which is ivcm in this case) and the directory (here ./locale) specified by the call gettext.install('ivcm', './locale', unicode=False). The wxPython system uses the wxstd.mo files.
= py2exe and gettext =
The following gives some sample py2exe setup files which are adapted for gettext .mo files.
* [[py2exeAndGettext]] what needs to be done to handle .mo files with py2exe
= Switching between Left to Right and Right to Left Languages =
For an application that must be internationalized into languages that read from right to left, such as Hebrew or Arabic, there are other considerations besides translating the strings. The layout of the GUI should be able to switch layout order as well. This is especially important for when options or controls follow a determined direction in the thought process, as having this backwards will confuse the user.
Thankfully, this can be done with rather easily, but the use of sizers is required. You should be using sizers and relative sizes for widgets if you are creating an In8l application anyway, so this shouldn't be a problem.
First we need to determine the language direction. Assume the ''language'' variable is a string of the ISO 639-1 code.
{{{#!python
right_left_languages = ('ar', 'dv', 'fa', 'ha', 'he', 'ps', 'ur', 'yi')
if language not in right_left_languages:
langLTR = True
alignment = wx.ALIGN_LEFT
else:
langLTR = False
alignment = wx.ALIGN_RIGHT
}}}
Now when the elements are set in place at creation time, we can use the ''langLeftToRight'' variable to determine the order in which they should be placed in the sizer:
{{{#!python
MrSizer = wx.BoxSizer(wx.HORIZONTAL)
# list containing the elements to place
# each element tuple contains:(element, proportion, padding)
SizerElements = [(self.repl_move,0,10),
(self.repl_move_pos,0,3),
(self.repl_move_pos_value,0,20),
(self.repl_move_txt,0,3),
((10,10),0,0), # don't forget about spacers!
(self.repl_move_txt_mod,0,3),
((10,10),0,0),
(self.staticText1,0,3),
(self.repl_move_txt_value,1,3),
(self.repl_move_txt_re,0,5),]
# left to right languages
if langLTR:
for i in SizerElements:
MrSizer.Add(i[0],i[1],wx.ALIGN_CENTER|wx.RIGHT,i[2])
# right to left languages:
else:
moveRowElements.reverse()
for i in SizerElements:
MrSizer.Add(i[0],i[1],wx.ALIGN_CENTER|wx.LEFT,i[2])
}}}
Sometimes, all that is necessary is changing the alignment, for example when a single element occupies an entire row. In this case the ''alignment'' variable suffices.
{{{#!python
MrsSizer = wx.BoxSizer(wx.VERTICAL)
MrsSizer.Add(self.staticText2,0,wx.ALL|alignment,5)
MrsSizer.Add(self.staticText3,0,wx.ALL|alignment,5)
}}}
= Examples =
== Example 1 ==
Here's an example I made while learning this. I hope it helps someone. -- Nate Silva
{{{#!python
"""
How to initialize the two translation systems: Python and wxWidgets.
"""
import sys, os
import gettext
import wx
# Hack to get the locale directory
basepath = os.path.abspath(os.path.dirname(sys.argv[0]))
localedir = os.path.join(basepath, "locale")
langid = wx.LANGUAGE_DEFAULT # use OS default; or use LANGUAGE_JAPANESE, etc.
domain = "messages" # the translation file is messages.mo
# Set locale for wxWidgets
mylocale = wx.Locale(langid)
mylocale.AddCatalogLookupPathPrefix(localedir)
mylocale.AddCatalog(domain)
# Set up Python's gettext
mytranslation = gettext.translation(domain, localedir,
[mylocale.GetCanonicalName()], fallback = True)
mytranslation.install()
if __name__ == '__main__':
# use Python's gettext
print _("Hello, World!")
# use wxWidgets' translation services
print wx.GetTranslation("Hello, World!")
# if getting unicode errors try something like this:
#print wx.GetTranslation("Hello, World!").encode("utf-8")
}}}
Alternately, to just run everything through the wxWidgets translation system, do something like this:
{{{#!python
mylocale = wx.Locale(langid)
mylocale.AddCatalogLookupPathPrefix(localedir)
mylocale.AddCatalog(domain)
_ = wx.GetTranslation
#if you are getting unicode errors, try something like:
#_ = lambda s: wx.GetTranslation(s).encode('utf-8')
}}}
== Example 2 ==
Ianaré Sévi, 2006/07/04
While the two examples above work flawlessly in Windows, when porting over to Mac OS X and Linux I noticed some problems. After much trial and error here is what should be a more complete solution. You will notice I am getting the language info from the file 'language.ini' - all it contains is an internal code for my application depending on what the user has set using another function. In this way language settings 'stick'. I am using the unicode version of wxPython, and this example has been tested successfully on win NT/2000/XP, Linux (Gnome), and Mac OS X86. The little Linux hack is to get things like calendars and stock buttons to display in the proper language.
{{{#!python
import wx
import os
import platform
import codecs
import sys
import gettext
def main():
# initialise language settings:
path = sys.path[0].decode(sys.getfilesystemencoding())
try:
langIni = codecs.open(os.path.join(path,u'language.ini'),'r', 'utf-8')
except IOError:
language = u'en' #defaults to english
pass
else:
language = langIni.read()
locales = {
u'en' : (wx.LANGUAGE_ENGLISH, u'en_US.UTF-8'),
u'es' : (wx.LANGUAGE_SPANISH, u'es_ES.UTF-8'),
u'fr' : (wx.LANGUAGE_FRENCH, u'fr_FR.UTF-8'),
}
mylocale = wx.Locale(locales[language][0], wx.LOCALE_LOAD_DEFAULT)
langdir = os.path.join(path,u'locale')
Lang = gettext.translation(u'messages', langdir, languages=[language])
Lang.install(unicode=1)
if platform.system() == 'Linux':
try:
# to get some language settings to display properly:
os.environ['LANG'] = locales[language][1]
except (ValueError, KeyError):
pass
#-> Code to launch application goes here. <-#
if __name__ == '__main__':
if 'unicode' not in wx.PlatformInfo:
print "\nInstalled version: %s\nYou need a unicode build of wxPython to run this application.\n"%version
else:
main()
}}}
- Also, use only ascii characters as your translatable strings (msgid) but use the unicode build of wxPython to display the strings returned by gettext (msgstr). This should allow support for all human languages by your application, however OS must include support as well (ie need to download Windows language pack to correctly view Japanese translation).
= Comments =
* Please feel free to provide any type of feedback on this page or on mki18n.py to me. I will be revamping mki18n.py soon. /Pierre Rouleau.
* You can email me at the address provided. I had so much spam at some point, so my email is encoded below.
* The encoded address is: 'zNbOpS.PfAxMe@bMjEgSrAaMuIgGnOcSz vA@PhInPrYyThHbOeNc'
* Use the following code to decode it:
{{{#!python
address[::-2].encode('nospam@31tor.com'[7:-4][::-1])
}}}
----------
Why you mix gettext and the wx.Locale class, just do it like this:
{{{#!python
def installwxgettext():
import wx
import __builtin__
__builtin__._ = wx.GetTranslation
}}}
this is the same way like the python gettext module it does, but more elegant i think...
----
Example 1 gives me a warning " wxStandardPathsBase::Get﴾﴿: create wxApp before calling this" on line containing "mylocale = wx.Locale(langid)" on Win7. I have a chicken and egg problem. I need this set up before making the wx app. At least I think so.
--------
= Additional Information =
For the non-ascii characters, save the text file with an utf-8 encoding.
For the translation or editing, you can use Poedit application.
It exist some version for Windows, Mac and Linux.
'''Link :'''
https://en.wikipedia.org/wiki/Poedit
https://fr.wikipedia.org/wiki/Poedit
[[https://wiki.wxpython.org/Tip%20of%20the%20Day%20frame%20(Phoenix)|https://wiki.wxpython.org/Tip%20of%20the%20Day%20frame%20%28Phoenix%29]]
[[https://wiki.wxpython.org/Internationalization%20-%20i18n%20(Phoenix)|https://wiki.wxpython.org/Internationalization%20-%20i18n%20%28Phoenix%29]]
- - - - -
https://wiki.wxpython.org/TitleIndex
https://docs.wxpython.org/