Djihed Afifi

I am dreaming about a single first stop next-gen (buzzword alert) website for all Open Source internationalisation. Complete with raw material downloading, translation allocation,  reviewing and quality assurance, statistics and error reporting, web based translation, downloading translation files, uploading translation files, and committing translation files. But before letting you know about my complete dream, let me preface that by stating what exists out there.

There are literally thousands of Open Source projects in various programming languages, from numerous developers and websites. Translation teams have come up with different ways of dealing with this variety to minimise time and maximise efficiency and translation reuse. Here are but a few software packages that deal with this chaos:

* Damned lies (as in lies, damned lies and statistics):

Available at: http://l10n.gnome.org

A statistics website that is largely passive. It reports translations statistics for GNOME (and Fedora), and has the most comprehensive reporting out there: it will tell you if you have errors in your po files,  it will list images that you need to capture for documentation, it will complain if you didn’t enable your language in the sources (LINGUAS). It has support for multiple version controls. It still remains a passive tool where translators can only look at statistics and not upload/change anything directly through it.

* Vertimus

Available at: http://gnomefr.traduc.org/suivi/

A file allocation and reviewing tool, used by the French GNOME team. This handy tools assigns multiple roles to users. Users can be assigned PO files that they can work on off-line. Uploaded translation can be reviewed as necessary. Project maintainer/coordinator has to commit files separately.

* Transifex

Available at: https://translate.fedoraproject.org/submit/

A system for submitting translations to upstream projects. It aims to support as many back-ends as possible.  Coordinators can submit a translation which is then passed on to upstream using an account created by Transifex. Projects thus have to be subscribed to benefit from this.

* Pootle

Available at: http://translate.sourceforge.net/wiki/pootle/index

Pootle is a web based translation tool. Behind Pootle is a powerful Python API for reading and writing translation files of multiple formats. Pootle’s interface, however, leaves a lot to be desired and its translations are not strictly version controlled (I could be wrong, but damage sometimes damage can be irreversible). It is file based so it can be real slow for big files.

* open-tran

Available at: http://open-tran.eu/

Open-tran aggregates strings from hundreds of packages and offers suggestions for strings that you search on it. It offers a web API so you can query for strings from your application if you want to.

* Rosetta

Available at (no source): https://launchpad.net/rosetta
Rosetta is probably the software with most features, and it still falls short in many places. It is a web based translation tool, with support for statistics and translation suggestions. It is not as good as Vertimus in its user and QA management, and it again can only deal with projects that are subscribed to it. Plus, its awkward status as a middle man between many upstream projects and ubuntu limits it greatly. It is also greatly limited by the fact that it is closed source.

There are also other in-development projects such as Verbatim of Mozilla.

Obviously, these solutions approach different and the same problems differently. Each with their own login system, their own database (if any), each in effect with their own website. In the end, if a translation team wanted package allocation, web based translation and a good commit back-end then they have to go to three different places, plus all the stuff that is usually provided upstream (KDE online statistics for example, Debian’s, etc). And that would not be enough because some of them only offer access to a small subset of packages.

—-

Now, my dream is a website that integrates the functionality of all of these tools into one inclusive web tool that offers all of these services. It is inclusive in that projects can be subscribed with or without upstream’s help (e.g Pootle). It is up to the language coordinators to decide whether they want to do their translation using the tool (e.g Pootle). The tool can commit on behalf of the coordinators with its own commit credentials or with the coordinator supplied ones if necessary (e.g Transifex partly). Translators can be allocated packages to translate, with complete quality assurance and reviewing steps (Vertimus). The tool displays comprehensive statistics and error messages about translation files if necessary (damned-lies). The tools offers web based translation (e.g Rosetta) with the option to download files and translate them locally if they wish to. The tool offers suggestions from a wide variety of packages (open-trans) with an option to enforce a per language terminology dictionary. Packages can be opened as free for all translation, or only users of team X can translate/review them. Open packages may invite inexperienced translators and inconsistency but should be reviewed thoroughly.

Plugins can be written to compile and serve translations. In fact, the website should offer an API with good extensibility - the API offers downloading files, settings up a local repository if you need to, serving statistics, committing files etc.. Thus, it should be easy to circumvent the whole tool all together and do things the old way if a translator/team desires so.

The website can be augmented by web 2.0 (err, another buzzword) features. Meta statistics can be posted (User X did y translations, z reviews. Language team A did y strings, etc.). This stuff may look juvenile but can be greatly fun and encourages competition between teams (just look at what Wikipedia statistics do). Notes or even threads can be attached to packages or strings if necessary.

I believe such a tool would be an empowering tool for translators, especially for languages with small teams. For many teams, the same people more or less translate most open source packages, so such a tool will be of great help. It would greatly help consistency and make translators work on actual translations instead of back-end coordination. Teams would no longer have to replicate their setup across many projects.

I largely steered away from implementation details as I would like to discuss the idea first. I believe the previously mentioned projects addressed many of the individual features separately so they are certainly demonstratively doable. What remains is to just bring the lot together and integrate it.

Any comments are welcome. I will post this content on various relevant mailing lists in a few days as well.

Here is a list of the new and usual packages for Minbar 0.2. New packages in this post are for Fedora and PCLinuxOS.

Ubuntu

A ubuntu package is available:

minbar_0.2.1-1_i386.deb

Debian

Minbar 0.2 is in Debian thanks to Adnène,
Minbar Debian packages.

(Scroll to the bottom and choose your architecture. Unless you know what you’re doing, yours is likely i386)

Fedora

Fedora packages have been built by Mohd Izhar Firdaus Ismail:

Fedora Minbar and ITL packages.

PCLinuxOS 2007

PCLinuxOS packages have been built by Suleman Khalid:

PCLinuxOS Minbar and ITL packages

ArchLinux AUR

ArchLinux packages have been built by linuxSEAT:

• libitl
• itools
• Minbar

(You need libitl for Minbar. itools is an optional package for command line prayer utilities).
Source

The source tarballs for 0.2.1 can be downloaded via these links (the same in different formats):

minbar-0.2.1.zip

minbar-0.2.1.tar.gz

minbar-0.2.1.tar.bz2

Other distributions

Packages for other distributions will be posted here as they become available. If you know how to build a package, please send it to djihed at gmail dot com.

Over the last few weeks we have maintained a healthy state of GNOME 2.22 translation to Arabic, thanks to Anas Husseini, Abou Manal, Ahmad Farghal, Osama Khayat and Khaled Hosny. Currently we have 95% done and we are at the third spot. Detailed statistics are listed below, and here is a link to the official GNOME statistics:

http://l10n.gnome.org/releases/gnome-2-22
The new modules for the next release have been settled, and the strings are more or less finalised, except for a few changes in the future. So, time to switch our focus to this release and get it done as soon as possible.

I have been quiet busy in the last few week with fixing various RTL and Arabic bugs in GNOME. Expect a few fixes in the next release. I will continue on this work so unfortunately I won’t have much time for translation.

It would be very good if we could complete this release as soon as possible. I would like to dedicate the last 2 weeks before the release to strict Quality Assurance and Translation Revision. I have built the whole new GNOME from sources and so I expect to test and review most translations.

Please feel free to assign yourself any of the uncompleted packages in this list, and let me know what you have taken.

* Translated    39311 95.42%
* Fuzzy          1271 3.09%
* Untranslated    615 1.49%

* Total         41197
* To be done     1886 4.58%

Incomplete Packages
--------------------
Package                 Translated Fuzzy Untranslated
eel                       30         0    1
libgnomekbd               49         0    1
nautilus                1161         1    0
metacity                 514         1    2
gnome-applets-locations 4355         3    0
gnome-terminal           483         3    0
gnome-desktop             65         0    5
gnome-applets            939         5    0
epiphany                 909         5    1
gnome-session            122         6    1
gtk -properties         1501         6    1
gnome-volume-manager     196         8    0
evince                   287         3    6
gtk-engines               32         4    6
cheese                    45         5    5
file-roller              249         7    3
sound-juicer             156         8    6
gnome-build              110         7   10
gnome-system-tools       231        11    6
evolution-data-server   1026        14    3
gtk                      908        16    1
gnome-utils              723        10    9
gnome-system-monitor     210         8   12
vino                      84         8   12
ekiga                    632        11   10
gconf                    453        24    0
eog                      245         8   17
gnome-keyring             59        16   15
totem                    426        22    9
tomboy                   355        24    9
gnome-power-manager      443        36    3
deskbar-applet           186        16   24
libgnome                 215        40    0
seahorse                 727        37    6
empathy                  266         7   37
gnome-control-center     829        41    9
gimmie                   122        40   13
yelp                     289        29   30
gdm                       66        51   15
gtksourceview            273        62   17
gdl                       17        48   33
gnome-games             1684        49   37
gcalctool                303        72   27
anjuta                  1853        80   32
orca                     924        63   52
glib                     215        56   68
gvfs                       0       149   29
evolution               4664       151   32

I have received many emails about Minbar’s prayer calculation method. First, one thing I would like to make sure everyone understands is that Minbar is not supposed to replace a real human Athan call. It is software, and it may have errors and bugs, so make really sure that you check your times with your local mosque and see if Minbar conforms with them. If they don’t, let me know.

Don’t forget to try to change the calculation method and see if the software produces correct times: you can change the calculation method in the second tab of the preferences dialog:

So far, the countries with the most problems are the far east countries: Indonesia and Malaysia. I have received many emails from these two countries. I would like to let you know that I acknowledge this problem and I will work on solving it. I take correct prayer times very seriously and will do my utmost to resolve it.

Minbar uses libitl for its prayer calculation. I have not written libitl, so I am planning to do a code overhaul and look at the correctness and authenticity of the implementation. This may take time, so I apologise for this in advance. In the mean time, feel free to always send me an email when you notice a calculation error; include in your report:

• Your city details: Altitude, Longitude, Latitude, name, country.
• Your country’s time zone.
• The times that Minbar gives you for your city.
• The date.
• Your country’s calculation method: if you know it.
• Your country’s Madhab.

Minbar 0.2 packages have been produced for ubuntu and debian. Please follow this link to my minbar 0.2 post to get them. They are linked at the end.

http://djihed.com/minbar/minbar-02-released

I am pleased to announce the release of Minbar 0.2. The last version is now Minbar 0.2.1 after a small update to fix a bug.
Minbar is the GNOME prayer times application.

Screenshot:

More at: http://files.djihed.com/minbarss/0.2

Amongst the bug fixes and new features are:

• Prayer Timetable (Calendar).
• Ability to start minimised to tray.
• Fix for .desktop installation.
• Store Athan path.
• City names database.
• Qibla Direction.
• Multiple Athan files, Subh Athan file as well.
• Appropriate plurals support.
• Add CLI option to start minimised.
• Better artwork.
• Accept non integer values for time zone for countries like India.
• Close now hides to tray.

Artwork by:

• Yulian Ardiansyah.

Updated and new translations:

• English.
• Arabic: Abderrahim Kitouni, Khaled Hosny.
• Bosnian: Nedim, Alma.
• Farsi: Mohammad Derakhshani
• French: Abdelmonam Kouka, Mohammed Adnène Trojette.
• Indonesian: Andika Triwidada.
• Pashto: Zabeeh Khan
• Spanish: Guillermo Garcia Rojas Covarrubias.
• Turkish: Mustafa Doğan.
• Urdu: Suleman Khalid.
• Urdu PK: Muhammad Ali Makki.

Special thanks to:

• Abderrahim Kitouni: for helping with integrating the city database and many other things.
• Mohammed Adnène Trojette: build files, testing.
• Ahmad Farghal: Testing.

Author:

• Djihed Afifi.

I would also like to thank all those who sent me comments and emails about Minbar, as well as the guys at the ubuntume and zekr projects.

Errors and bugs in Minbar are, of course, solely my responsibility.
Contact:

• For any issues, please comment here or contact djihed at gmail dot com.

Download

Ubuntu

A ubuntu package is available:

minbar_0.2.1-1_i386.deb

Debian

Minbar 0.2 is in Debian thanks to Adnène,
Minbar Debian packages.

(Scroll to the bottom and choose your architecture. Unless you know what you’re doing, yours is likely i386)

Fedora

Fedora packages have been built by Mohd Izhar Firdaus Ismail:

Fedora Minbar and ITL packages.

PCLinuxOS
PCLinuxOS packages have been built by Suleman Khalid:

PCLinuxOS Minbar and ITL packages

Source

The source tarballs for 0.2.1 can be downloaded via these links (the same in different formats):

minbar-0.2.1.zip

minbar-0.2.1.tar.gz

minbar-0.2.1.tar.bz2

Other distributions

Packages for other distributions will be posted here as they become available. If you know how to build a package, please send it to djihed at gmail dot com.

A while back I was interviewed by Leonardo Fontenelle (An active Free Software l10n contributor from Brazil), it is worth mentioning here. There, I described many aspects of our work in Arabic translation and Arabeyes. Here I quote some passages:

On the open nature of the translation process:

I guess I don’t like some particular phase per see, but all in all, I very much adore the open nature of it. Right from the source code to the actual compiled message catalogues. You can’t really get much more open that this. This openness really pays off when translating weird messages, when trying to view translations live, when comparing with translations of other packages, when viewing translations of different languages, etc.

On the Technical Dictionary:

The technical dictionary is basically an English-Arabic dictionary for computing terms. At first, we started making it with .po files, but that created many problems with versioning and discussing the terms. So I had the idea of uploading the terms to our Wiki. I used some scripts to convert the .po files to Wiki xml input. The wiki, being open, allows people to edit as they see fit, discuss terms, suggest alternatives, etc. Then finally, there are some scripts that take the wiki pages and convert them back to .po files, as well as .pdf suitable for printing/reading. The experience was very rewarding to us.

On contributors:

Finally, minbar 0.2 is nearing release. What is left is just a bit of testing and translations.

First, if you would like to translate Minbar then now is the right time. Get this file and start translating, then send it back to djihed at gmail dot com.
Minbar.pot

For detailed instructions on translating, please see end of post.
Also, if you have the ability to package software for any distro (.rpm, .deb etc) than I’d love to hear from you.

Thanks to Yulian Ardiansyah, we now have a new shiny icon and logo:

And this is a screenshot of the new version, emphasis has been placed on elegance.

Amongst the features and the fixes of the new version are:

• A new Qibla direction feature.
• City selection using a database of cities.
• Ability to specify different Athan files for Subh and other times
• Ability to specify half hour GMT correction for countries like India
• Ability to start minimised.

As always, the code resides in Arabeyes CVS.

If you have never translated a .po for .pot file before, here is what you need to do:

• Download the file: Minbar.pot, right click and choose save.
• Install poedit: a software that helps you translate these files. You can either install it using your distribution’s repository (Linux) or you can download it: Download poedit.
• Open the file minbar.pot with poedit. Make sure to fill up both “settings” and “preferences”. For encoding, choose “UTF-8″.
• Translate the file. When you are done, save it and send it to my address: djihed at gmail dot com.

NOTE1: Minbar 0.2 has already been translated to Arabic, Indonesian, Spanish, French, Urdu (Pakistan), Turkish, Bosnian and Pashto.
NOTE2: Those wishing to translate it to Norweigian or Farsi can update the preexisting 0.1 translations: Norweigian: no.po , Farsi: fa.po

I know that it has been a long time since I showed up here! Unfortunately, over the last months I have been way too busy. End of my final university year (I’m graduating today!), job hunting, some vacation, and going for Umra. I will be back in a few weeks inchallah.

Apologies to all concerned, and to all those who have sent me emails and comments. Rest assured that I still have your messages and that I will take the time to go through them in detail. Minbar 2.0 is missing only a few more tweaks then it will be ready. Also, salam to all those at Arabeyes! I still continue to monitor the Wiki, and I always get amazed by the ongoing participation.

Soon inchallah.

About 1 month ago I wrote some scripts to get the technical dictionary contents from the Wiki to a pdf.

At the time I wondered how many people would download it, so I did not spend a good time to make it neatly formatted. The end result pdf was not very good.

Today, however, I decided to check if people are actually downloading it. Doing some Data Mining on the Apache logs, I was quite surprised to see 387 downloads, 197 are from unique IP addresses. The break down of unique downloaders by country is shown below.

Encouraging, time to go back, beautify it and make it look good.

On another front, parsing the referers (which page directed people to the pdf), about 70% were from the arabic page, 30% from the English page. This highlights the importance of having pages in both languages for Arabic and English speakers.

Breakdown of unique Technical Dictionary downloads by country:

38 : EG, Egypt
22 : US, United States
12 : SA, Saudi Arabia
10 : GB, United Kingdom
9 : PS, Palestine
9 : DZ, Algeria
8 : TR, Turkey
8 : AE, United Arab Emirates
7 : MA, Morocco
6 : JO, Jordan
6 : DE, Germany
5 : IL, Israel
3 : TN, Tunisia
3 : QA, Qatar
3 : OM, Oman
3 : --, N/A
3 : IT, Italy
3 : FR, France
3 : CZ, Czech Republic
2 : SD, Sudan
2 : LY, Libyan Arab Jamahiriya
2 : KW, Kuwait
2 : IN, India
2 : FI, Finland
2 : CN, China
2 : BH, Bahrain
2 : A2, Satellite Provider
1 : ZA, South Africa
1 : UA, Ukraine
1 : TH, Thailand
1 : SY, Syrian Arab Republic
1 : RU, Russian Federation
1 : PK, Pakistan
1 : NZ, New Zealand
1 : NO, Norway
1 : MG, Madagascar
1 : LB, Lebanon
1 : HK, Hong Kong
1 : ES, Spain
1 : BG, Bulgaria
1 : AU, Australia
1 : AL, Albania

I would like to announce a new version of tclgeoip, the TCL extension for GeoIP.

Amongst the changes in 0.2:

1. Fixed a segfault when loading databases.
2. Introduced a new function to check presence of databases (db_avail)
3. Updated documentation.

Grab the .tar.gz sources here. Also, sources from SVN repository.

A debian package is being cooked.

At Arabeyes we have several Open Source Projects for translation, totaling more than 300000 strings. Our biggest challenge is preserving consistency and correctness across all of these projects. From experience, while some of the words seem obvious in English, their counterparts in another langauge (such as Arabic) can sparke heated debates. A while back, in an effort to tackle this, we introduced the Arabic Technical Computing Dictionary, and we hosted it on a Wiki for open discussion by any translator. A few scripts extract the messages every week into various neat formats for translators, including .csv, .po and even a .pdf that is suitable for printing (I still need to fix some issues the latter).

However, we still had problems prioritising discussions: which words should we discuss first? which need immediate attention? are we missing any important words? are we over analysing words that are not important? I believe these are important questions that every translation project to any language should consider. They are especially very important for languages that do not yet have a concise and established list of terminology translations.

The solution seems quite obvious: analysis of existing projects. While computers are quite bad at translation with human level accuracy, they are extremely good at statistics and counting. So why not exploit that?

So I put together a number of scripts that analyse .po files and output statistical data that can help us answer the previous questions. I operated on the biggest four open source projects we have: KDE, GNOME, OpenOffice.org and Mozilla (including Firefox and Thunderbird), the string pool had nearly 300000 strings. Reading the .po files, the scripts count the number of occurences of each word. The top 10 most used words* are:

1. 4734 file
2. 3002 name
3. 2538 error
4. 2268 text
5. 2110 use
6. 1946 list
7. 1931 window
8. 1869 select
9. 1826 open
10. 1825 show

Again this list may seem obvious, but a word like “select” has a few equivalents in Arabic, and we struggled to agree to one term. The complete list is available in this file. A .pot [0.5 MB] template is also available, but beware that it contains a lot of rubbish, and there are nearly 20000 entries so I can’t clean it all. If you clean it, I’d be interested in having a copy.

This only gives us that most popular words. We also want the most popular technical dictionary entries (including combinations such as “system administrator”, the previous list contains only singular words). The most important technical dictionary entries are in this list.

The difference between the complete list and the technical dictionary gives us the list of words that are not in the technical dictionary. Many of them are very important, I was honestly surprised to see words like “toolbar” and tab” not being in the wiki.

Analysis of individual projects is also available. Here are the most popular words for KDE, GNOME, Mozilla and OpenOffice.org.

The complete set of scripts and results reside in Arabeyes CVS. feel free to make use of them. The scripts are GPL but the data follows the license of the individual projects**. If you have a different way of analysis, or have another set of words from your language I would be very interested in hearing from you.

Special thanks to Chahibi for helping me with some ideas.

* KDE was excluded because bash complained of too many files (arguments). If you know of a way to increase the limit please let me know.
** I believe they are comptaible with the GPL. If you disagree, please send me an email (no need to yell).

The first release of Minbar was somewhat hurried, and was more intended for personal use, and as a test of whether Muslim Gnome users really want something like this.

It turns out a lot of people are happy about the idea, and there is an acceptable level of adoption amongst Gnome users. Thanks everyone! This encouraged me to work together a better Minbar for next release, and may be, just may be, build it and distribute for Windows.
So this is meant to get some ideas from people on what they want. Currently I am thinking of:

• City names list, choosing a city instead of entering its details.
• Hide window on start-up (Already in CVS).
• Windows port.
• Fix the menu icon.
• Package it for deb and rpm distributions.
• …? (Your idea here)

Any more?

Alriyadh, a popular Saudi newspaper were kind enough to do an interview with me about CDMaftooh.

I took the opportunity to voice concerns about the Arabic Open Source movement, and voice out that we do actually need governmental/educational help to spread the word.

They published the interview in their 13th March 2007 issue. I’d love to get a hard copy from any of you my Saudi brothers! The server took a hard beating on that day, I thought It wouldn’t make it through, but it did.

Read the interview, It is in Arabic.

Time for a blatant me me post.

Finally my application to the Gnome Foundation has been accepted, I am now an official Gnome Foundation Member!

Thanks to Adnene Minbar is in debian experimental. It is not yet in testing because of dependancy issues which should be solved when the debian guys update some packages.

Get the .deb for debian based distributions. I would also very much appreciate it if someone could package it for Fedora, Suse and other rpm based distros.

Finally Arabic Gnom 2.16 has been fully translated to Arabic, thanks to a dedicated team of translators.

See statistics!

We will be emphasising more on quality and correctness for gnome 2.18, since most of the job is already done. Some details are available in this Wiki page.

Minbar, the Gnome Prayer Application is finally done. This first release does the Athan call (prayer call) and notifies before the time of the prayer. It uses ITL for prayer calculation, which has a high level of accuracy. It is initially available in Arabic, English and French.

Download the source tarball from here. You need a few libraries to build it, the enclosed README lists them. If you have any problems, I will be happy to know about them. There is a debian .deb to be released soon for debian derived distributions. If you can package this for your own distribution that would be great!

This screenshot shows it with its tray icon (the Kaaba shaped icon). Click for a larger version.

Many thanks to Abderrahim Kitouni and Mohammed Adnène Trojette for the great help!

The technical dictionary aims to translate and standardise technical terms that are used in software. It is an effort to unify the terms used across all projects, to present the user with consistant and understandable interfaces.

We have been, since some time, trying to discuss the terms using the mailing list. This created many problems: discussions are forgotten, people discuss terms over and over and there is no single point of reference for all terms.

To solve these, we have recently imported the dictionary to the wiki. You are welcome to contribute, whether you are a native english or bilingual speaker. Proficiency is not need, normal users are also welcome since the work is being done for them, you can comment on whether the term is understandable to you.
The dictionary is available here (Currently only words starting with A are there):

http://wiki.arabeyes.org/Technical_Dictionary

Leave a comment on the discussion page if you would like guidance on where to contribute.

---

NOTE: It has come to my attention that many people are being redirect here from Google for Minbar. The latest about Minbar is available in this entry: Minbar.

---

Gnome is sorely missing Islamic/Arabic applications. There has been some attempts in the distant past, like a gnome applet called “akemsalatak” (cool name). Unfortunaly it won’t build, and I have not been able to establish contact with its maintainer.So here is a new Gnome Prayer Application. It is still very much a work in progress, I welcome any suggestions or contributions. The code resides in Arabeyes CVS here.

One of the cool libraries it uses it libglade. Libglade makes GUI’s so easy to make it’s almost a sin. The application uses gstreamer for Athan playback, and libitl (the Arabeyes islamic tools) to calculate the times.

Pre-Beta screenshot:

UPDATE: Minbar 0.1 has been released, please bookmark this link for the latest details: Minbar