Using Git and separating translations into their own l10n-LL repository

[Simos Xenitellis <simos lists googlemail com>]

Hi All,

This is a long-ish post regarding the migration to Git and
what we can do to make l10n a bit better.

Here I suggest to use the 'git submodule' support
so that the translation material for each language
reside in a single repository.
Comments would be appreciated.
If all is fine, I'll put this, with more details, to live.gnome.org.

When splitting the l10n files from each module, there is a choice to either
1. create a companion repository for each module (for example, for
mousetweaks, create 'mousetweaks-l10n')
that will hold all localisation files for all languages, for this module.
If we have 1000 modules, there would be 1000 additional companion l10n modules.
2. create a repository for each language, and this repository will
contain all localisation files for all modules.
If we have 1000 modules, there would be just 160 additional l10n
repositories (it's one new repository per language).

The right choice appears to be to create one repository per language.
There are many reasons which can be discussed if deemed necessary.

The rest of the e-mail shows how the separated repositories look like.
I used as an example the mousetweaks and vinagre modules, for the el,
es, fr and sv languages.
Both have help/ and po/ subdirectories with l10n material.
You can fork the generated (six) repositories from
http://github.com/simos/ if you want to try them out.

STRUCTURE (l10n-LL)
A language repository name is of the form 'l10n-LL', where LL is the
ISO 639 (-123) language code as usual.
Inside 'l10n-LL' there are directories per module (with the module
name), and further subdirectories 'po/' and 'help/' as necessary.
For example,

l10n-el
├─ mousetweaks
│   ├─ help
│   │   └─ el
│   │       └─ el.po
│   └─ po
│       └─ el.po
└─ vinagre
    ├─ help
    │   └─ el
    │       └─ el.po
    └─ po
        └─ el.po

and

l10n-es
├─ mousetweaks
│   ├─ help
│   │   └─ es
│   │       ├─ es.po
│   │       └─ figures
│   │           ├─ mouse-a11y-add-applet-to-panel-window.png
│   │           ├─ mouse-a11y-dwell-checkbox.png
│   │           ├─ mouse-a11y-dwell-click-type-applet.png
│   │           ├─ mouse-a11y-dwell-click-type-window.png
│   │           ├─ mouse-a11y-dwell-ctw-checkbox.png
│   │           ├─ mouse-a11y-dwell-delay-slider.png
│   │           ├─ mouse-a11y-dwell-gesture-mapping.png
│   │           ├─ mouse-a11y-dwell-mode-choice.png
│   │           ├─ mouse-a11y-dwell-motion-treshold.png
│   │           ├─ mouse-a11y-pointer-capture-context-menu.png
│   │           ├─ mouse-a11y-pointer-capture-locked.png
│   │           ├─ mouse-a11y-pointer-capture-preferences.png
│   │           ├─ mouse-a11y-ssc-checkbox.png
│   │           ├─ mouse-a11y-ssc-delay-slider.png
│   │           └─ mouse-a11y-tab.png
│   └─ po
│       └─ es.po
└─ vinagre
    ├─ help
    │   └─ es
    │       ├─ es.po
    │       └─ figures
    │           └─ vinagre-screenshot.png
    └─ po
        └─ es.po

STRUCTURE ('l10n' supermodule)
Per git parlance, we create a 'l10n' supermodule, and inside it we add
each language repository as submodules.
Thus,

l10n
├─ README
├─ l10n-el/
├─ l10n-es/
├─ l10n-fr/
└─ l10n-sv/

The above graphic shows the first level of directories.
When we add a new language, the l10n coordinators will add a new entry
here to the new repository 'l10n-LL'.
Each 'l10n-LL' entry is added with 'git submodule add', and points to
a repository created earlier.

STRUCTURE (module)
Now, each module (such as mousetweaks and vinagre) need to simply add
the 'l10n' supermodule.
We remove from help/ and po/ all *.po and figures/ files. For our two
modules, the po/ and help/ subdirectories would look like

mousetweak:
po
├─ LINGUAS
├─ POTFILES.in
└─ POTFILES.skip
help
├─ C
│   ├─ figures
│   │   ├─ mouse-a11y-dwell-checkbox.png
│   │   ├─ mouse-a11y-dwell-click-type-applet.png
│   │   ├─ mouse-a11y-dwell-click-type-window.png
│   │   ├─ mouse-a11y-dwell-ctw-checkbox.png
│   │   ├─ mouse-a11y-dwell-delay-slider.png
│   │   ├─ mouse-a11y-dwell-gesture-mapping.png
│   │   ├─ mouse-a11y-dwell-mode-choice.png
│   │   ├─ mouse-a11y-dwell-motion-treshold.png
│   │   ├─ mouse-a11y-pointer-capture-context-menu.png
│   │   ├─ mouse-a11y-pointer-capture-locked.png
│   │   ├─ mouse-a11y-pointer-capture-preferences.png
│   │   ├─ mouse-a11y-ssc-checkbox.png
│   │   ├─ mouse-a11y-ssc-delay-slider.png
│   │   └─ mouse-a11y-tab.png
│   ├─ legal.xml
│   └─ mousetweaks.xml
├─ Makefile.am
└─ mousetweaks.omf.in

vinagre:
po
├─ LINGUAS
├─ POTFILES.in
└─ POTFILES.skip
help
├─ C
│   ├─ figures
│   │   └─ vinagre-screenshot.png
│   ├─ legal.xml
│   └─ vinagre.xml
├─ Makefile.am
└─ vinagre.omf.in

(mental hint: the C/ folder should actually move to the 'l10n'
supermodule, in 'l10n-C')
We have removed the ChangeLog files as the commits are moved to the submodules.
We have to figure out what to do LINGUAS. Is it possible to remove altogether?

EXAMPLE
This assumes you have the 'git' package ('git-core' if you use
Debian/Ubuntu) installed.

A. Cloning the repository of the module (when building from source, etc)
$ git clone git://github.com/simos/mousetweaks.git
Initialized empty Git repository in /tmp/GIT/mousetweaks/.git/
remote: Counting objects: 89, done.
remote: Compressing objects: 100% (88/88), done.
remote: Total 89 (delta 9), reused 0 (delta 0)
Receiving objects: 100% (89/89), 288.43 KiB | 96 KiB/s, done.
Resolving deltas: 100% (9/9), done.
$ cd mousetweaks
$ git submodule init
Submodule 'l10n' (git://github.com/simos/l10n.git) registered for path 'l10n'
$ git submodule update
Initialized empty Git repository in /tmp/GIT/mousetweaks/l10n/.git/
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (6/6), done.
Submodule path 'l10n': checked out 'd7f6178ccf9fc8f65c6c42ee9724a65649ad044b'
$ cd l10n
$ ls
README
$ git branch
* (no branch)
  master
$ git checkout master
Previous HEAD position was d7f6178... Initial import.
Switched to branch "master"
$ ls
l10n-el/  l10n-es/  l10n-fr/  l10n-sv/  README            (the
directories are still empty)
$ git submodule init
Submodule 'l10n-el' (git://github.com/simos/l10n-el.git) registered
for path 'l10n-el'
Submodule 'l10n-es' (git://github.com/simos/l10n-es.git) registered
for path 'l10n-es'
Submodule 'l10n-fr' (git://github.com/simos/l10n-fr.git) registered
for path 'l10n-fr'
Submodule 'l10n-sv' (git://github.com/simos/l10n-sv.git) registered
for path 'l10n-sv'
$ git submodule update
Initialized empty Git repository in /tmp/GIT/t/mousetweaks/l10n/l10n-el/.git/
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (5/5), done.
Receiving objects: 100% (8/8), 15.41 KiB, done.
remote: Total 8 (delta 0), reused 0 (delta 0)
Submodule path 'l10n-el': checked out '3fded9f61797faea2c3596d96af5de75b442146f'
... (this command brings in the files for all repositories)
$ _

At this stage we managed to expand the repository and bring in all the files.
Of course, the translation files are in different locations, so the
building scripts
and tools like intltool would need to be updated.

B. Cloning an individual language repository (this is what translators
would need to do).
$ git clone git://github.com/simos/l10n-el.git
Initialized empty Git repository in /tmp/GIT/l10n-el/.git/
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 8 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (8/8), 15.41 KiB, done.
$ _

Now, in /tmp/GIT/l10n-el/ I have all the translation files. I can edit
vinagre/po/el.po and

$ git commit -a -m "Updated Greek translation"
[master]: created d19db87: "Updated Greek translation"
 1 files changed, 1 insertions(+), 1 deletions(-)
$ _

and finally push to the GNOME git server,

$ git push
Counting objects: 9, done.
Delta compression using 2 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 440 bytes, done.
Total 5 (delta 1), reused 0 (delta 0)
To git github com:simos/l10n-el.git
   3fded9f..b8975fc  master -> master
$ _

To top it up, here is a list of advantages
1. Developers can simply add the 'l10n' supermodule as a submodule to
their projects.
When they clone their repositories, the translation files are not
downloaded by default.
You need to run 'git submodule init' and then 'git submodule update', etc.
Thus, it saves time and space.
2. Translators can keep a local offline version of their 'l10n-LL' repository.
This is quite small in size. The translators that wish to, can simply grab
the new versions of the PO files from l10n.gnome.org, thus would not
need to clone all modules.
3. There is a process for translators who want to clone the modules
and perform the manual steps.
4. Tools such as Pootle and Transifex will be able to work better
because they would not demand to hold full repositories of all
modules.

And some disadvantages
1. It would require to make invasive changes across all modules.
Building scripts and tools
would need to adapt.
2. It gives too much freedom for errors to the l10n teams. Each l10n
repository needs to conform to the right directory structure so that
the files are picked up when we compile.

Simos