Re: Using Git and separating translations into their own l10n-LL repository

[Simos Xenitellis <simos lists googlemail com>]

On Tue, Jan 13, 2009 at 11:31 PM, Gil Forcada <gforcada gnome org> wrote:
> Hi Simos,
>
> Great explanation, a couple of questions:
>
> - There would be any way to have in a single folder the code and
> translations (aka, the way is right now)? There are some translators
> (I'm counting in) that likes to have the code also.

I am not sure which module you have in mind.
I would say that the idea is, if the translation files are expected to
be managed by damned-lies/l10n.gnome.org, they should reside in the
l10n-LL submodule.

> - The same concern about maintainers forgetting to git pull(?)
> translators is expanded, now they should git pull(?) every single git
> repository (if 160 languages, 160 git pulls(?) before each release).

The 'git submodule update' command, when you run in from the 'l10n'
supermodule,
pulls all the l10n-LL repositories automatically by default.
There is an option to 'git submodule update' to pull individual
language repositories as well.

> Also, if the new DL will have commit functionality (maybe in half a
> year, Claude ... :) and then *all* translations will be committed this
> way, wouldn't be overkill to have all this system?
>
> I mean, if it's a temporal situation that, shouldn't be more productive
> to instead of "wasting" time doing this git submodule thing to provide
> resources and time to get commit functionality to DL that seems to be
> more desired by translators (that's my guess no data actually, just
> supposed so my arguments seems better :D).

I would consider that separating code from localisation files would be
a fundamental improvement rather than a temporary solution.
The issue of manpower to make such changes is the main disadvantage.

It would be easier to provide commit support to damned-lies when
we have separated l10n-LL repositories. Damned-lies would 'auto'-commit
only to designated repositories.

I do not know the details of the SVN hooks and damned-lies. I think
that if a commit/push to a project would call a hook that would invoke
'intltool-update -P' (create POT template file) and store it
somewhere, then damned-lies would not need to clone locally any other
GNOME modules.

To add a few more advantages,
1. Can add access controls for translators to the l10n-LL repositories.
2. It allows translators to make changes across all translations (for
example, change 'widget' in all my translations of GNOME).
3. Pootle could be adapted to perform easier GNOME translation marathons.
4. GTranslator could work as KBabel, it would clone the l10n-LL
repository and would be able to show all translations in a huge sorted
list. This way, similar translations can be easily identified.

>
> El dt 13 de 01 de 2009 a les 23:01 +0000, en/na Simos Xenitellis va
> escriure:
>> Hi All,
>>
>> This is a long-ish post regarding the migration to Git and
>> what we can do to make l10n a bit better.
>>
>> Here I suggest to use the 'git submodule' support
>> so that the translation material for each language
>> reside in a single repository.
>> Comments would be appreciated.
>> If all is fine, I'll put this, with more details, to live.gnome.org.
>>
>> When splitting the l10n files from each module, there is a choice to either
>> 1. create a companion repository for each module (for example, for
>> mousetweaks, create 'mousetweaks-l10n')
>> that will hold all localisation files for all languages, for this module.
>> If we have 1000 modules, there would be 1000 additional companion l10n modules.
>> 2. create a repository for each language, and this repository will
>> contain all localisation files for all modules.
>> If we have 1000 modules, there would be just 160 additional l10n
>> repositories (it's one new repository per language).
>>
>> The right choice appears to be to create one repository per language.
>> There are many reasons which can be discussed if deemed necessary.
>>
>> The rest of the e-mail shows how the separated repositories look like.
>> I used as an example the mousetweaks and vinagre modules, for the el,
>> es, fr and sv languages.
>> Both have help/ and po/ subdirectories with l10n material.
>> You can fork the generated (six) repositories from
>> http://github.com/simos/ if you want to try them out.
>>
>> STRUCTURE (l10n-LL)
>> A language repository name is of the form 'l10n-LL', where LL is the
>> ISO 639 (-123) language code as usual.
>> Inside 'l10n-LL' there are directories per module (with the module
>> name), and further subdirectories 'po/' and 'help/' as necessary.
>> For example,
>>
>> l10n-el
>> ├─ mousetweaks
>> │   ├─ help
>> │   │   └─ el
>> │   │       └─ el.po
>> │   └─ po
>> │       └─ el.po
>> └─ vinagre
>>     ├─ help
>>     │   └─ el
>>     │       └─ el.po
>>     └─ po
>>         └─ el.po
>>
>> and
>>
>> l10n-es
>> ├─ mousetweaks
>> │   ├─ help
>> │   │   └─ es
>> │   │       ├─ es.po
>> │   │       └─ figures
>> │   │           ├─ mouse-a11y-add-applet-to-panel-window.png
>> │   │           ├─ mouse-a11y-dwell-checkbox.png
>> │   │           ├─ mouse-a11y-dwell-click-type-applet.png
>> │   │           ├─ mouse-a11y-dwell-click-type-window.png
>> │   │           ├─ mouse-a11y-dwell-ctw-checkbox.png
>> │   │           ├─ mouse-a11y-dwell-delay-slider.png
>> │   │           ├─ mouse-a11y-dwell-gesture-mapping.png
>> │   │           ├─ mouse-a11y-dwell-mode-choice.png
>> │   │           ├─ mouse-a11y-dwell-motion-treshold.png
>> │   │           ├─ mouse-a11y-pointer-capture-context-menu.png
>> │   │           ├─ mouse-a11y-pointer-capture-locked.png
>> │   │           ├─ mouse-a11y-pointer-capture-preferences.png
>> │   │           ├─ mouse-a11y-ssc-checkbox.png
>> │   │           ├─ mouse-a11y-ssc-delay-slider.png
>> │   │           └─ mouse-a11y-tab.png
>> │   └─ po
>> │       └─ es.po
>> └─ vinagre
>>     ├─ help
>>     │   └─ es
>>     │       ├─ es.po
>>     │       └─ figures
>>     │           └─ vinagre-screenshot.png
>>     └─ po
>>         └─ es.po
>>
>> STRUCTURE ('l10n' supermodule)
>> Per git parlance, we create a 'l10n' supermodule, and inside it we add
>> each language repository as submodules.
>> Thus,
>>
>> l10n
>> ├─ README
>> ├─ l10n-el/
>> ├─ l10n-es/
>> ├─ l10n-fr/
>> └─ l10n-sv/
>>
>> The above graphic shows the first level of directories.
>> When we add a new language, the l10n coordinators will add a new entry
>> here to the new repository 'l10n-LL'.
>> Each 'l10n-LL' entry is added with 'git submodule add', and points to
>> a repository created earlier.
>>
>> STRUCTURE (module)
>> Now, each module (such as mousetweaks and vinagre) need to simply add
>> the 'l10n' supermodule.
>> We remove from help/ and po/ all *.po and figures/ files. For our two
>> modules, the po/ and help/ subdirectories would look like
>>
>> mousetweak:
>> po
>> ├─ LINGUAS
>> ├─ POTFILES.in
>> └─ POTFILES.skip
>> help
>> ├─ C
>> │   ├─ figures
>> │   │   ├─ mouse-a11y-dwell-checkbox.png
>> │   │   ├─ mouse-a11y-dwell-click-type-applet.png
>> │   │   ├─ mouse-a11y-dwell-click-type-window.png
>> │   │   ├─ mouse-a11y-dwell-ctw-checkbox.png
>> │   │   ├─ mouse-a11y-dwell-delay-slider.png
>> │   │   ├─ mouse-a11y-dwell-gesture-mapping.png
>> │   │   ├─ mouse-a11y-dwell-mode-choice.png
>> │   │   ├─ mouse-a11y-dwell-motion-treshold.png
>> │   │   ├─ mouse-a11y-pointer-capture-context-menu.png
>> │   │   ├─ mouse-a11y-pointer-capture-locked.png
>> │   │   ├─ mouse-a11y-pointer-capture-preferences.png
>> │   │   ├─ mouse-a11y-ssc-checkbox.png
>> │   │   ├─ mouse-a11y-ssc-delay-slider.png
>> │   │   └─ mouse-a11y-tab.png
>> │   ├─ legal.xml
>> │   └─ mousetweaks.xml
>> ├─ Makefile.am
>> └─ mousetweaks.omf.in
>>
>> vinagre:
>> po
>> ├─ LINGUAS
>> ├─ POTFILES.in
>> └─ POTFILES.skip
>> help
>> ├─ C
>> │   ├─ figures
>> │   │   └─ vinagre-screenshot.png
>> │   ├─ legal.xml
>> │   └─ vinagre.xml
>> ├─ Makefile.am
>> └─ vinagre.omf.in
>>
>> (mental hint: the C/ folder should actually move to the 'l10n'
>> supermodule, in 'l10n-C')
>> We have removed the ChangeLog files as the commits are moved to the submodules.
>> We have to figure out what to do LINGUAS. Is it possible to remove altogether?
>>
>> EXAMPLE
>> This assumes you have the 'git' package ('git-core' if you use
>> Debian/Ubuntu) installed.
>>
>> A. Cloning the repository of the module (when building from source, etc)
>> $ git clone git://github.com/simos/mousetweaks.git
>> Initialized empty Git repository in /tmp/GIT/mousetweaks/.git/
>> remote: Counting objects: 89, done.
>> remote: Compressing objects: 100% (88/88), done.
>> remote: Total 89 (delta 9), reused 0 (delta 0)
>> Receiving objects: 100% (89/89), 288.43 KiB | 96 KiB/s, done.
>> Resolving deltas: 100% (9/9), done.
>> $ cd mousetweaks
>> $ git submodule init
>> Submodule 'l10n' (git://github.com/simos/l10n.git) registered for path 'l10n'
>> $ git submodule update
>> Initialized empty Git repository in /tmp/GIT/mousetweaks/l10n/.git/
>> remote: Counting objects: 6, done.
>> remote: Compressing objects: 100% (5/5), done.
>> remote: Total 6 (delta 0), reused 0 (delta 0)
>> Receiving objects: 100% (6/6), done.
>> Submodule path 'l10n': checked out 'd7f6178ccf9fc8f65c6c42ee9724a65649ad044b'
>> $ cd l10n
>> $ ls
>> README
>> $ git branch
>> * (no branch)
>>   master
>> $ git checkout master
>> Previous HEAD position was d7f6178... Initial import.
>> Switched to branch "master"
>> $ ls
>> l10n-el/  l10n-es/  l10n-fr/  l10n-sv/  README            (the
>> directories are still empty)
>> $ git submodule init
>> Submodule 'l10n-el' (git://github.com/simos/l10n-el.git) registered
>> for path 'l10n-el'
>> Submodule 'l10n-es' (git://github.com/simos/l10n-es.git) registered
>> for path 'l10n-es'
>> Submodule 'l10n-fr' (git://github.com/simos/l10n-fr.git) registered
>> for path 'l10n-fr'
>> Submodule 'l10n-sv' (git://github.com/simos/l10n-sv.git) registered
>> for path 'l10n-sv'
>> $ git submodule update
>> Initialized empty Git repository in /tmp/GIT/t/mousetweaks/l10n/l10n-el/.git/
>> remote: Counting objects: 8, done.
>> remote: Compressing objects: 100% (5/5), done.
>> Receiving objects: 100% (8/8), 15.41 KiB, done.
>> remote: Total 8 (delta 0), reused 0 (delta 0)
>> Submodule path 'l10n-el': checked out '3fded9f61797faea2c3596d96af5de75b442146f'
>> ... (this command brings in the files for all repositories)
>> $ _
>>
>> At this stage we managed to expand the repository and bring in all the files.
>> Of course, the translation files are in different locations, so the
>> building scripts
>> and tools like intltool would need to be updated.
>>
>> B. Cloning an individual language repository (this is what translators
>> would need to do).
>> $ git clone git://github.com/simos/l10n-el.git
>> Initialized empty Git repository in /tmp/GIT/l10n-el/.git/
>> remote: Counting objects: 8, done.
>> remote: Compressing objects: 100% (5/5), done.
>> remote: Total 8 (delta 0), reused 0 (delta 0)
>> Receiving objects: 100% (8/8), 15.41 KiB, done.
>> $ _
>>
>> Now, in /tmp/GIT/l10n-el/ I have all the translation files. I can edit
>> vinagre/po/el.po and
>>
>> $ git commit -a -m "Updated Greek translation"
>> [master]: created d19db87: "Updated Greek translation"
>>  1 files changed, 1 insertions(+), 1 deletions(-)
>> $ _
>>
>> and finally push to the GNOME git server,
>>
>> $ git push
>> Counting objects: 9, done.
>> Delta compression using 2 threads.
>> Compressing objects: 100% (4/4), done.
>> Writing objects: 100% (5/5), 440 bytes, done.
>> Total 5 (delta 1), reused 0 (delta 0)
>> To git github com:simos/l10n-el.git
>>    3fded9f..b8975fc  master -> master
>> $ _
>>
>> To top it up, here is a list of advantages
>> 1. Developers can simply add the 'l10n' supermodule as a submodule to
>> their projects.
>> When they clone their repositories, the translation files are not
>> downloaded by default.
>> You need to run 'git submodule init' and then 'git submodule update', etc.
>> Thus, it saves time and space.
>> 2. Translators can keep a local offline version of their 'l10n-LL' repository.
>> This is quite small in size. The translators that wish to, can simply grab
>> the new versions of the PO files from l10n.gnome.org, thus would not
>> need to clone all modules.
>> 3. There is a process for translators who want to clone the modules
>> and perform the manual steps.
>> 4. Tools such as Pootle and Transifex will be able to work better
>> because they would not demand to hold full repositories of all
>> modules.
>>
>> And some disadvantages
>> 1. It would require to make invasive changes across all modules.
>> Building scripts and tools
>> would need to adapt.
>> 2. It gives too much freedom for errors to the l10n teams. Each l10n
>> repository needs to conform to the right directory structure so that
>> the files are picked up when we compile.
>>
>> Simos
>> _______________________________________________
>> gnome-i18n mailing list
>> gnome-i18n gnome org
>> http://mail.gnome.org/mailman/listinfo/gnome-i18n
> --
> gil forcada
>
> [ca] guifi.net - una xarxa lliure que no para de créixer
> [en] guifi.net - a non-stopping free network
> bloc: http://gil.badall.net
>
>