Re: Fwd: Current state of Python bindings.

[Mark Doffman <mark doffman codethink co uk>]

On Tue, 2007-08-28 at 21:50 +0200, Philip Van Hoof wrote:
> On Tue, 2007-08-28 at 15:18 +0100, Mark Doffman wrote:
> 
> > I have been looking at helping out with the Python bindings for
> > Tinymail, to try and put a Road-map together for their completion.
> > Currently the Python bindings have lapsed a little and don't seem to
> > be functional.
> 
> That's great and indeed very needed. As you know would for example the
> OLPC project be very interested in such high quality bindings.
> 
> > I have been unable to load the tinymail-camel library, and as such get
> > the Python-demo going. The tinymail-camel library has difficulty with
> > the TinySessionCamel and TinyStreamCamel. As these are types based on
> > CamelObject, and internal, they may need to be removed from the
> > interface generation.
> 
> Sounds like a fair trade off. They are only to be used internally with
> the exception of the tny_camel_account_set_session API and the
> tny_session_camel_set_initialized API.
> 
> Maybe a solution for those two can be put together? Usually (always)
> there will be only one TnySessionCamel instance.
> 
> For TnyStreamCamel, this is the opposite of the TnyCamelStream. Camel
> has its own abstract stream type called CamelStream. TnyStreamCamel
> converts any CamelStream to Tinymail's abstract TnyStream type. The
> TnyStreamCamel is an adaptor that adapts CamelStream to TnyStream.
> 
> This type is only to be used internally (but could be used externally,
> if you need it in your own implementations or Tinymail-enhancements).

This still needs some thinking about on my part, but there is an 
existing mechanism supported by PyGObject that could work here. GBoxed 
is used by PyGtk to wrap structures into the GObject type system, which
is then wrapped into the Python Type system. I'll try and follow this 
method with CamelSession and CamelStream. 

> 
> 
> > Philip has indicated that there are a few more issues lurking beneath
> > this before the demo code works properly, probably related to the
> > incomplete wrapping of the Tinymail interface mentioned below.
> 
> > Nevertheless the first step must be to get the demo code going, and
> > I'll be looking to do that soon. 
> 
> Sounds like a good goal.
> 
> > Completing the basic API:
> > 
> > Presumably as Tinymail has moved on, the .override files used to put
> > together the bindings have become incomplete.
> 
> In TnyFolder I think two or three new methods that need wrapping got
> added. The existing ones now all have different prototypes and use the
> TnyStatus and TnyStatusCallback rather than their own status-callback
> and status integers. The existing ones are a simple port and the new
> ones will, I guess, look a lot like the ported existing ones.
> 
> > There are a number of methods and functions that are not wrapped. An
> > overview of the parts of Tinymail and the methods that still need
> > wrapping are below: 
> > 
> > tinymail
> > ----------
> > Old style constructors(4)
> > Unwrapped methods(30)
> > Unwrapped functions(3) 
> > Unknown arguments(24)
> > 
> > tinymailui-gtk
> > -----------------
> > Old style constructors(4)
> > Unwrapped functions(2) 
> > Unknown arguments(1)
> > 
> > tinymail-camel
> > -------------------
> > Old style constructors(3)
> > Unwrapped methods(6)
> > Unwrapped functions(2)
> > Unknown arguments(6)
> > 
> > (Old style constructors warning is an issue with subclassing types
> > that means if signals or parameters are defined with the new type only
> > gobject__init__ can be used in python constructor. Probably not a big
> > issue.) 
> > 
> > There are a number of easy fixes amongst the functions that still need
> > wrapping, such as class methods that have been wrongly allocated as
> > module functions - these can be fixed by small changes to
> > the .override file. There are many functions that will have to be
> > wrapped by hand, such as those requiring a callback / returning values
> > via their arguments. I envisage that completing the API so
> > pygtk-codegen has 100% coverage is a medium body of work encompassing
> > perhaps a couple of man weeks 
> 
> 
> > API Test coverage - Nothings complete until its tested. So much of the
> > wrapping is automatically generated and presumed correct - but we will
> > need to test the interface as much as possible. Perhaps the
> > libtinymail-tests could be converted to python to exercise the
> > interfaces? 
> 
> Or add a new test suite that is specialised for this task? The unit test
> are a bit outdated as well, and I'm also planning to create a few Qemu
> virtual machines that can be used with the standard Tinymail tests.
> 
> I wouldn't want to throw away the plain-C tests in favour of Python
> ones. I would prefer to add a Python-binding test suite in stead.

I think for the moment I will add tests for any Function, method and
type bindings that I add manually in bindings/python/tests.

Your right, the tests would hopefully be specialized to look for
problems in the bindings. I'm not expert enough to say what this would 
look like right now. I may Skim the PyGtk bugs to see what they have run
into. 

> 
> > Extensions to the basic API -
> > 
> > The basic Python API would consist of the work done above, Tinymail
> > classes are available in Python for use. PyGObject and the
> > pygtk-codegen tools have the ability to make the virtual functions of
> > a GType available for use in Python. This would allow python code to,
> > for example, implement a TnyMsg interface and use the wrapped
> > TnyGtkMsgView class to display that message. As a framework for
> > developing applications I think that this is a fairly essential
> > feature. 
> 
> Agree
> 
> > I have been unable to build the python bindings with virtual methods
> > using scanvirtuals.py. The tool creates errors in the .defs file -
> > multiple declarations of the same method.
> 
> I noticed the same problem. I fear that either scanvirtuals.py or pygtk
> needs to get bugfixed for this first.
> 
> > Completing this stage would involve checking that all virtual
> > functions, in interfaces or otherwise are, properly wrapped. 
> 
> Correct
> 
> > Pythonification -
> > This is something that will beautify the bindings done once they are
> > essentially complete. (And a stage PyGtk seems to have been going
> > through for a while). 
> 
> Pre note: this is something I personally know little about. 
> 
> > Fields - People using the Python bindings may wish to be able to
> > access some fields of an object directly. (eg widget.window) There
> > seems to be methods for doing this already in the PyGtk code
> > generators. This may be a case of changing the .defs file. It requires
> > some thought from someone who knows the interfaces well to decide
> > WHICH fields are suitable for simple access. 
> 
> ok
> 
> > Iterators - There is other perhaps more difficult work for example,
> > python provides its own iterator API, and interface for type
> > implementation of this. How does it fit in with the TinyMail iterator
> > interface?
> 
> I think one person once started this work. It's available in one of the
> existing .override files.

Yes, It looks like some work has been started to get a TnyList to
implement the sequence protocol in Python. I can't tell how complete
this is yet but some functions are still missing. 

> 
> > Docs - Provide meaningful doc strings. 
> > 
> > I need some help sorting this all out into something with more near
> > term goals and a time line. I think that the python bindings to be
> > considered complete and usable should be declaring virtual functions.
> > They also need decent testing, either by unit testing, or completing a
> > mail client that stretches the interface to the extent we want. 
> 
> Agree
> 

Thanks

Mark