Re: PyGObject (3.4.0) - importing overrides modules from different directories

[Lukas Vacek <lucas vacek gmail com>]

https://bugzilla.gnome.org/show_bug.cgi?id=687335

On Wed, Oct 31, 2012 at 10:13 PM, Simon Feltman <s feltman gmail com> wrote:
> Hi Lukas,
>
> Yes, and my apologies, I did not read your original email close enough
> to realize this is what you are running into. Before going the route
> of symlinks it may be worth while to see if .pkg files or
> pkg_resources.declare_namespace could help the situation (I haven't
> had the time to look into either). We could also start to turn
> gi/overrides/ into a pure namespace package by removing code from the
> __init__.py (I think we should do this in any case) but this will take
> a deprecation cycle. The following PEP is also informative if you
> haven't gone through it already:
> http://www.python.org/dev/peps/pep-0420/
> At this point, creating a bug for tracking this so we can start
> trading and reviewing patches more easily would be nice.
>
> -Simon
>
> On Wed, Oct 31, 2012 at 3:35 AM, Lukas Vacek <lucas vacek gmail com> wrote:
>> Simon,
>>
>> I see you had ran into the problem with extend_path and code in
>> __init__.py already (comment 26 in the bug report you posted earlier).
>> Do you think it is a good practice for pygobject overrides module
>> developers to create a symlink from __init__.py in their installed
>> module to __init__.py in pygobject overrides/ directory?
>>
>> Lukas
>>
>> On Fri, Oct 26, 2012 at 1:25 PM, Lukas Vacek <lucas vacek gmail com> wrote:
>>> Hey Simon,
>>>
>>> Thanks for pointing me to the bug report. Yes, it is possible to load
>>> overrides from different paths already. This goes a bit further and
>>> allows loading overrides from different paths no matter the order of
>>> sys.path.
>>>
>>> Let's have a look at gst-python as it is the subject of the bug
>>> report: if you run PYTHONPATH=path-to-gst-python python -c "import gi;
>>> print(str(dir(gi)))" you can see gi module wasn't loaded properly.
>>> That's because __init__.py in pygobject overrides contains other code
>>> than just calling extend_path from pkgutil. Indeed you can use
>>> PYTHONPATH=path-to-pygobject:path-to-gst-python and all is good.
>>>
>>> This might seem like a trivial issue. Well, it's nothing major, but it
>>> causes confusion - for example, the two issues in pitivi I linked in
>>> my first email stems from this.
>>>
>>> Cheers,
>>> Lukas
>>>
>>> On Fri, Oct 26, 2012 at 12:53 PM, Simon Feltman <s feltman gmail com> wrote:
>>>> Hi Lukas,
>>>>
>>>> You should already be able to use overrides in different paths
>>>> searchable by python (anything that shows up in sys.path). The trick
>>>> is you need to use pkgutil within each of the __init__.py files within
>>>> your custom gi/overrides folders. There are more details which may be
>>>> a helpful in this bug report:
>>>> https://bugzilla.gnome.org/show_bug.cgi?id=680913
>>>>
>>>> -Simon
>>>>
>>>> On Fri, Oct 26, 2012 at 4:23 AM, Lukas Vacek <lucas vacek gmail com> wrote:
>>>>> I see the last mail didn't catch much attention. Anyway, I have come
>>>>> up with two possible solutions to the problem.
>>>>>
>>>>> 1) Use imp instead of __import__
>>>>> Just look for a file "gi/overrides/Module_name.py" in sys.path and
>>>>> don't worry about __init__.py (overrides module can't use __init__.py
>>>>> anyway as they clash with pygobject's overrides' init which is the
>>>>> root cause of the whole problem), then load it with imp. It feels a
>>>>> bit untidy but works fine. This is the attached
>>>>> pygobject-3.4.0-overrides-load-py-files-instead-of-modules.patch.
>>>>> Indeed this should look for other supported suffixes than .py as well.
>>>>>
>>>>> 2) Ask overrides module developers to symlink __init__.py
>>>>> This solution feels cleaner and requires no changes to existing
>>>>> pygobject code. The solution is to link __init__.py from overrides
>>>>> modules to system pygobject overrides directory in case overrides
>>>>> module is installed outside of pygobject tree. With autotools, this
>>>>> can be done easily by using install-exec-hook -
>>>>> gst-python-symlink-__init__.py-to-system-gi-overrides-on-install.patch
>>>>> demonstrates this idea on gst-python.
>>>>>
>>>>> Cheers,
>>>>> Lukas Vacek
>>>>>
>>>>> On Mon, Oct 22, 2012 at 1:35 PM, Lukas Vacek <lucas vacek gmail com> wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> I have found out it's not possible to load installed GI overrides from
>>>>>> a different directory than where PyGObject itself is installed
>>>>>> (/usr/lib/python2.7/dist-packages/ in my case).
>>>>>>
>>>>>> Ideally gi overrides modules could be installed to user site-packages
>>>>>> directory and still would be loaded by system installation of
>>>>>> PyGObject.
>>>>>>
>>>>>> The problem is that the overrides module can't install __init__.py
>>>>>> file because it would conflict with __init__.py already installed by
>>>>>> system PyGObject. (Or even if the file was not installed in the same
>>>>>> location but the installed overrides module (with own __init__.py)
>>>>>> would appear before system PyGObject in sys.path it would be loaded
>>>>>> instead of PyGObject as gi module and break things. )
>>>>>>
>>>>>> However, __init__.py must be present in the gi overrides module
>>>>>> directory in order to be recognized by __import__ call made by
>>>>>> PyGObject when loading overrides module (module.py around line 244).
>>>>>>
>>>>>> This makes it complicated to use gi overrides module from a different
>>>>>> directory. The workaround is to load _uninstalled_ gi overrides
>>>>>> modules with PYTHONPATH which explicitly mentions PyGObject path
>>>>>> before the path to overrides modules. For example, this is how I run
>>>>>> pitivi: PYTHONPATH=/usr/lib/python2.7/dist-packages/:~/dev/gst-python/
>>>>>> python bin/pitivi
>>>>>>
>>>>>> Pitivi bugs https://bugzilla.gnome.org/show_bug.cgi?id=686341 and
>>>>>> https://bugzilla.gnome.org/show_bug.cgi?id=686342 seem to be related
>>>>>> to/caused by this.
>>>>>>
>>>>>> To solve this pygobject might look for a ".py" file instead of a module.
>>>>>>
>>>>>> Thanks!
>>>>>> Lukas Vacek
>>>>>
>>>>> _______________________________________________
>>>>> python-hackers-list mailing list
>>>>> python-hackers-list gnome org
>>>>> https://mail.gnome.org/mailman/listinfo/python-hackers-list
>>>>>