Re: Using Git and separating translations into their own l10n-LL repository

[Simos Xenitellis <simos lists googlemail com>]

On Wed, Jan 14, 2009 at 3:34 PM, Gil Forcada <gforcada gnome org> wrote:
> El dc 14 de 01 de 2009 a les 00:42 +0000, en/na Simos Xenitellis va
> escriure:
>> On Tue, Jan 13, 2009 at 11:31 PM, Gil Forcada <gforcada gnome org> wrote:
>> > Hi Simos,
>> >
>> > Great explanation, a couple of questions:
>> >
>> > - There would be any way to have in a single folder the code and
>> > translations (aka, the way is right now)? There are some translators
>> > (I'm counting in) that likes to have the code also.
>>
>> I am not sure which module you have in mind.
>
> Sorry, I wasn't really clear. I meant to keep like it is now with code
> and translations together.
>
>> I would say that the idea is, if the translation files are expected to
>> be managed by damned-lies/l10n.gnome.org, they should reside in the
>> l10n-LL submodule.
>>
>> > - The same concern about maintainers forgetting to git pull(?)
>> > translators is expanded, now they should git pull(?) every single git
>> > repository (if 160 languages, 160 git pulls(?) before each release).
>>
>> The 'git submodule update' command, when you run in from the 'l10n'
>> supermodule,
>> pulls all the l10n-LL repositories automatically by default.
>> There is an option to 'git submodule update' to pull individual
>> language repositories as well.
>
> If I understand correctly the repository will be like:
> l10n
> - l10n-LL
> -- evolution
> --- po
> ---- LL.po
> -- eog
> --- po
> ---- LL.po
> ...
>
> So if they update the whole l10n module (the top module) they will get
> all translations from all modules not all translations from their
> module.

Indeed, that is an issue. I did not find an easy way so that the
'l10n' tree can stay outside the repositories,
and each module would have a symbolic link towards 'l10n'.
That is,

 mousetweaks
 ├─ l10n (link to ../l10n)
 │...

In addition to this, there is the issue with branching; all modules in
a GNOME release
should branch at the same time, and have a standard common branch name.

Cheers,
Simos

>
>> > Also, if the new DL will have commit functionality (maybe in half a
>> > year, Claude ... :) and then *all* translations will be committed this
>> > way, wouldn't be overkill to have all this system?
>> >
>> > I mean, if it's a temporal situation that, shouldn't be more productive
>> > to instead of "wasting" time doing this git submodule thing to provide
>> > resources and time to get commit functionality to DL that seems to be
>> > more desired by translators (that's my guess no data actually, just
>> > supposed so my arguments seems better :D).
>>
>> I would consider that separating code from localisation files would be
>> a fundamental improvement rather than a temporary solution.
>> The issue of manpower to make such changes is the main disadvantage.
>>
>> It would be easier to provide commit support to damned-lies when
>> we have separated l10n-LL repositories. Damned-lies would 'auto'-commit
>> only to designated repositories.
>>
>> I do not know the details of the SVN hooks and damned-lies. I think
>> that if a commit/push to a project would call a hook that would invoke
>> 'intltool-update -P' (create POT template file) and store it
>> somewhere, then damned-lies would not need to clone locally any other
>> GNOME modules.
>>
>> To add a few more advantages,
>> 1. Can add access controls for translators to the l10n-LL repositories.
>> 2. It allows translators to make changes across all translations (for
>> example, change 'widget' in all my translations of GNOME).
>> 3. Pootle could be adapted to perform easier GNOME translation marathons.
>> 4. GTranslator could work as KBabel, it would clone the l10n-LL
>> repository and would be able to show all translations in a huge sorted
>> list. This way, similar translations can be easily identified.
>>
>> >
>> > El dt 13 de 01 de 2009 a les 23:01 +0000, en/na Simos Xenitellis va
>> > escriure:
>> >> Hi All,
>> >>
>> >> This is a long-ish post regarding the migration to Git and
>> >> what we can do to make l10n a bit better.
>> >>
>> >> Here I suggest to use the 'git submodule' support
>> >> so that the translation material for each language
>> >> reside in a single repository.
>> >> Comments would be appreciated.
>> >> If all is fine, I'll put this, with more details, to live.gnome.org.
>> >>
>> >> When splitting the l10n files from each module, there is a choice to either
>> >> 1. create a companion repository for each module (for example, for
>> >> mousetweaks, create 'mousetweaks-l10n')
>> >> that will hold all localisation files for all languages, for this module.
>> >> If we have 1000 modules, there would be 1000 additional companion l10n modules.
>> >> 2. create a repository for each language, and this repository will
>> >> contain all localisation files for all modules.
>> >> If we have 1000 modules, there would be just 160 additional l10n
>> >> repositories (it's one new repository per language).
>> >>
>> >> The right choice appears to be to create one repository per language.
>> >> There are many reasons which can be discussed if deemed necessary.
>> >>
>> >> The rest of the e-mail shows how the separated repositories look like.
>> >> I used as an example the mousetweaks and vinagre modules, for the el,
>> >> es, fr and sv languages.
>> >> Both have help/ and po/ subdirectories with l10n material.
>> >> You can fork the generated (six) repositories from
>> >> http://github.com/simos/ if you want to try them out.
>> >>
>> >> STRUCTURE (l10n-LL)
>> >> A language repository name is of the form 'l10n-LL', where LL is the
>> >> ISO 639 (-123) language code as usual.
>> >> Inside 'l10n-LL' there are directories per module (with the module
>> >> name), and further subdirectories 'po/' and 'help/' as necessary.
>> >> For example,
>> >>
>> >> l10n-el
>> >> ├─ mousetweaks
>> >> │   ├─ help
>> >> │   │   └─ el
>> >> │   │       └─ el.po
>> >> │   └─ po
>> >> │       └─ el.po
>> >> └─ vinagre
>> >>     ├─ help
>> >>     │   └─ el
>> >>     │       └─ el.po
>> >>     └─ po
>> >>         └─ el.po
>> >>
>> >> and
>> >>
>> >> l10n-es
>> >> ├─ mousetweaks
>> >> │   ├─ help
>> >> │   │   └─ es
>> >> │   │       ├─ es.po
>> >> │   │       └─ figures
>> >> │   │           ├─ mouse-a11y-add-applet-to-panel-window.png
>> >> │   │           ├─ mouse-a11y-dwell-checkbox.png
>> >> │   │           ├─ mouse-a11y-dwell-click-type-applet.png
>> >> │   │           ├─ mouse-a11y-dwell-click-type-window.png
>> >> │   │           ├─ mouse-a11y-dwell-ctw-checkbox.png
>> >> │   │           ├─ mouse-a11y-dwell-delay-slider.png
>> >> │   │           ├─ mouse-a11y-dwell-gesture-mapping.png
>> >> │   │           ├─ mouse-a11y-dwell-mode-choice.png
>> >> │   │           ├─ mouse-a11y-dwell-motion-treshold.png
>> >> │   │           ├─ mouse-a11y-pointer-capture-context-menu.png
>> >> │   │           ├─ mouse-a11y-pointer-capture-locked.png
>> >> │   │           ├─ mouse-a11y-pointer-capture-preferences.png
>> >> │   │           ├─ mouse-a11y-ssc-checkbox.png
>> >> │   │           ├─ mouse-a11y-ssc-delay-slider.png
>> >> │   │           └─ mouse-a11y-tab.png
>> >> │   └─ po
>> >> │       └─ es.po
>> >> └─ vinagre
>> >>     ├─ help
>> >>     │   └─ es
>> >>     │       ├─ es.po
>> >>     │       └─ figures
>> >>     │           └─ vinagre-screenshot.png
>> >>     └─ po
>> >>         └─ es.po
>> >>
>> >> STRUCTURE ('l10n' supermodule)
>> >> Per git parlance, we create a 'l10n' supermodule, and inside it we add
>> >> each language repository as submodules.
>> >> Thus,
>> >>
>> >> l10n
>> >> ├─ README
>> >> ├─ l10n-el/
>> >> ├─ l10n-es/
>> >> ├─ l10n-fr/
>> >> └─ l10n-sv/
>> >>
>> >> The above graphic shows the first level of directories.
>> >> When we add a new language, the l10n coordinators will add a new entry
>> >> here to the new repository 'l10n-LL'.
>> >> Each 'l10n-LL' entry is added with 'git submodule add', and points to
>> >> a repository created earlier.
>> >>
>> >> STRUCTURE (module)
>> >> Now, each module (such as mousetweaks and vinagre) need to simply add
>> >> the 'l10n' supermodule.
>> >> We remove from help/ and po/ all *.po and figures/ files. For our two
>> >> modules, the po/ and help/ subdirectories would look like
>> >>
>> >> mousetweak:
>> >> po
>> >> ├─ LINGUAS
>> >> ├─ POTFILES.in
>> >> └─ POTFILES.skip
>> >> help
>> >> ├─ C
>> >> │   ├─ figures
>> >> │   │   ├─ mouse-a11y-dwell-checkbox.png
>> >> │   │   ├─ mouse-a11y-dwell-click-type-applet.png
>> >> │   │   ├─ mouse-a11y-dwell-click-type-window.png
>> >> │   │   ├─ mouse-a11y-dwell-ctw-checkbox.png
>> >> │   │   ├─ mouse-a11y-dwell-delay-slider.png
>> >> │   │   ├─ mouse-a11y-dwell-gesture-mapping.png
>> >> │   │   ├─ mouse-a11y-dwell-mode-choice.png
>> >> │   │   ├─ mouse-a11y-dwell-motion-treshold.png
>> >> │   │   ├─ mouse-a11y-pointer-capture-context-menu.png
>> >> │   │   ├─ mouse-a11y-pointer-capture-locked.png
>> >> │   │   ├─ mouse-a11y-pointer-capture-preferences.png
>> >> │   │   ├─ mouse-a11y-ssc-checkbox.png
>> >> │   │   ├─ mouse-a11y-ssc-delay-slider.png
>> >> │   │   └─ mouse-a11y-tab.png
>> >> │   ├─ legal.xml
>> >> │   └─ mousetweaks.xml
>> >> ├─ Makefile.am
>> >> └─ mousetweaks.omf.in
>> >>
>> >> vinagre:
>> >> po
>> >> ├─ LINGUAS
>> >> ├─ POTFILES.in
>> >> └─ POTFILES.skip
>> >> help
>> >> ├─ C
>> >> │   ├─ figures
>> >> │   │   └─ vinagre-screenshot.png
>> >> │   ├─ legal.xml
>> >> │   └─ vinagre.xml
>> >> ├─ Makefile.am
>> >> └─ vinagre.omf.in
>> >>
>> >> (mental hint: the C/ folder should actually move to the 'l10n'
>> >> supermodule, in 'l10n-C')
>> >> We have removed the ChangeLog files as the commits are moved to the submodules.
>> >> We have to figure out what to do LINGUAS. Is it possible to remove altogether?
>> >>
>> >> EXAMPLE
>> >> This assumes you have the 'git' package ('git-core' if you use
>> >> Debian/Ubuntu) installed.
>> >>
>> >> A. Cloning the repository of the module (when building from source, etc)
>> >> $ git clone git://github.com/simos/mousetweaks.git
>> >> Initialized empty Git repository in /tmp/GIT/mousetweaks/.git/
>> >> remote: Counting objects: 89, done.
>> >> remote: Compressing objects: 100% (88/88), done.
>> >> remote: Total 89 (delta 9), reused 0 (delta 0)
>> >> Receiving objects: 100% (89/89), 288.43 KiB | 96 KiB/s, done.
>> >> Resolving deltas: 100% (9/9), done.
>> >> $ cd mousetweaks
>> >> $ git submodule init
>> >> Submodule 'l10n' (git://github.com/simos/l10n.git) registered for path 'l10n'
>> >> $ git submodule update
>> >> Initialized empty Git repository in /tmp/GIT/mousetweaks/l10n/.git/
>> >> remote: Counting objects: 6, done.
>> >> remote: Compressing objects: 100% (5/5), done.
>> >> remote: Total 6 (delta 0), reused 0 (delta 0)
>> >> Receiving objects: 100% (6/6), done.
>> >> Submodule path 'l10n': checked out 'd7f6178ccf9fc8f65c6c42ee9724a65649ad044b'
>> >> $ cd l10n
>> >> $ ls
>> >> README
>> >> $ git branch
>> >> * (no branch)
>> >>   master
>> >> $ git checkout master
>> >> Previous HEAD position was d7f6178... Initial import.
>> >> Switched to branch "master"
>> >> $ ls
>> >> l10n-el/  l10n-es/  l10n-fr/  l10n-sv/  README            (the
>> >> directories are still empty)
>> >> $ git submodule init
>> >> Submodule 'l10n-el' (git://github.com/simos/l10n-el.git) registered
>> >> for path 'l10n-el'
>> >> Submodule 'l10n-es' (git://github.com/simos/l10n-es.git) registered
>> >> for path 'l10n-es'
>> >> Submodule 'l10n-fr' (git://github.com/simos/l10n-fr.git) registered
>> >> for path 'l10n-fr'
>> >> Submodule 'l10n-sv' (git://github.com/simos/l10n-sv.git) registered
>> >> for path 'l10n-sv'
>> >> $ git submodule update
>> >> Initialized empty Git repository in /tmp/GIT/t/mousetweaks/l10n/l10n-el/.git/
>> >> remote: Counting objects: 8, done.
>> >> remote: Compressing objects: 100% (5/5), done.
>> >> Receiving objects: 100% (8/8), 15.41 KiB, done.
>> >> remote: Total 8 (delta 0), reused 0 (delta 0)
>> >> Submodule path 'l10n-el': checked out '3fded9f61797faea2c3596d96af5de75b442146f'
>> >> ... (this command brings in the files for all repositories)
>> >> $ _
>> >>
>> >> At this stage we managed to expand the repository and bring in all the files.
>> >> Of course, the translation files are in different locations, so the
>> >> building scripts
>> >> and tools like intltool would need to be updated.
>> >>
>> >> B. Cloning an individual language repository (this is what translators
>> >> would need to do).
>> >> $ git clone git://github.com/simos/l10n-el.git
>> >> Initialized empty Git repository in /tmp/GIT/l10n-el/.git/
>> >> remote: Counting objects: 8, done.
>> >> remote: Compressing objects: 100% (5/5), done.
>> >> remote: Total 8 (delta 0), reused 0 (delta 0)
>> >> Receiving objects: 100% (8/8), 15.41 KiB, done.
>> >> $ _
>> >>
>> >> Now, in /tmp/GIT/l10n-el/ I have all the translation files. I can edit
>> >> vinagre/po/el.po and
>> >>
>> >> $ git commit -a -m "Updated Greek translation"
>> >> [master]: created d19db87: "Updated Greek translation"
>> >>  1 files changed, 1 insertions(+), 1 deletions(-)
>> >> $ _
>> >>
>> >> and finally push to the GNOME git server,
>> >>
>> >> $ git push
>> >> Counting objects: 9, done.
>> >> Delta compression using 2 threads.
>> >> Compressing objects: 100% (4/4), done.
>> >> Writing objects: 100% (5/5), 440 bytes, done.
>> >> Total 5 (delta 1), reused 0 (delta 0)
>> >> To git github com:simos/l10n-el.git
>> >>    3fded9f..b8975fc  master -> master
>> >> $ _
>> >>
>> >> To top it up, here is a list of advantages
>> >> 1. Developers can simply add the 'l10n' supermodule as a submodule to
>> >> their projects.
>> >> When they clone their repositories, the translation files are not
>> >> downloaded by default.
>> >> You need to run 'git submodule init' and then 'git submodule update', etc.
>> >> Thus, it saves time and space.
>> >> 2. Translators can keep a local offline version of their 'l10n-LL' repository.
>> >> This is quite small in size. The translators that wish to, can simply grab
>> >> the new versions of the PO files from l10n.gnome.org, thus would not
>> >> need to clone all modules.
>> >> 3. There is a process for translators who want to clone the modules
>> >> and perform the manual steps.
>> >> 4. Tools such as Pootle and Transifex will be able to work better
>> >> because they would not demand to hold full repositories of all
>> >> modules.
>> >>
>> >> And some disadvantages
>> >> 1. It would require to make invasive changes across all modules.
>> >> Building scripts and tools
>> >> would need to adapt.
>> >> 2. It gives too much freedom for errors to the l10n teams. Each l10n
>> >> repository needs to conform to the right directory structure so that
>> >> the files are picked up when we compile.
>> >>
>> >> Simos
>> >> _______________________________________________
>> >> gnome-i18n mailing list
>> >> gnome-i18n gnome org
>> >> http://mail.gnome.org/mailman/listinfo/gnome-i18n
>> > --
>> > gil forcada
>> >
>> > [ca] guifi.net - una xarxa lliure que no para de créixer
>> > [en] guifi.net - a non-stopping free network
>> > bloc: http://gil.badall.net
>> >
>> >
> --
> gil forcada
>
> [ca] guifi.net - una xarxa lliure que no para de créixer
> [en] guifi.net - a non-stopping free network
> bloc: http://gil.badall.net
>
>