Re: Python Docs (was Re: Coordination for developer documentations)

[Christoph Reiter <reiter christoph gmail com>]

On Mon, Mar 10, 2014 at 4:27 AM, Simon Feltman <s feltman gmail com> wrote:
> On Sat, Mar 8, 2014 at 9:15 AM, Christoph Reiter
> <reiter christoph gmail com> wrote:
> > On Sat, Mar 8, 2014 at 4:04 AM, Simon Feltman <s feltman gmail com> wrote:
> > > * GI function argument interpretation for Python docs would be as
> > > close as possible to pygobject by having the argument translation code
> > > living in pygobject itself. Preferably we could expose pygobjects
> > > internal argument caches which already have all the translation logic
> > > applied and re-use this for the docstrings. This has the benefit of
> > > lowering maintenance cost by ensuring the documentation for function
> > > arguments is in lockstep with how pygobject actually works.
> > >
> > > * Overrides and Python specific API extensions would automatically be
> > > included in the docs.
> >
> > (This is mainly the reason why I think Python introspection + Sphinx is
> > better in the long run than something like g-ir-doctool.. for the Python
> > bindings at least)
> >
> > This should be possible with the current Sphinx based approach but still
> > needs some work (it only shows the correct override signature atm). I need
> > to handle the following cases somehow:
> >
> > * In case the functionality of the overidden function differs from the
> > original one it should be possible to replace the gir documentation
> > all together.
>
> In terms of overrides having additional side effects beyond the
> intended GI API, I really hope we can deprecate those scenarios or
> additional arguments that invoke new/alternate behavior (like
> Gtk.MessageBox's "buttons" argument). It doesn't seem like there are
> very many of these cases and I'm not sure if the deprecations would help
> ease documentation or makes it more complex though. But I wouldn't
> mind leaving doc strings for deprecated functionality out granted
> usage of the function called as specified by a GI generated doc string
> works in a backwards compatible way (similarly I think we should
> ignore arguments/doc overrides for deprecated GObject initializers).

I was mostly thinking of functions returning "(ok, result)" which gets
translated to "result or None". For other stuff, please keep in mind that
documentation can also be used for understanding old code, so I wouldn't
just ignore and hide them. I'd prefer deprecation warnings in the docstring
with instructions how to do it right in those cases.

> > * In case the override just adds some additional arguments or default
> > values it should be possible to include the gir docs in the override
> > docstring (something like "%INCLUDE%" which gets replaced)
>
> That sounds like a good idea. A bit of bike-shedding, but I think a
> more explicit name would be nice. Something like
> "%gi_doc_string_body%" or variations on that which may or may not
> include argument documentation?

Sure.

> > * In case the override uses "*args, *kwargs" because of backwards
> > compatibility or because argument processing is delegated to some helper
> > function it should be possible to define a Python signature which replaces
> > the real one as seen by Sphinx.
>
> Do you mean the doc generator could figure this out or that we make
> sure to manually add doc strings in these cases?

No, that needs be done manually somehow. Sphinx already supports such
function signature overrides in docstrings (for C extensions I guess), but
that clashes with the signature pgi exposes for type information, so I need
to find another way.

> > Once I have this working with pgi, I will try to move the override changes
> > needed to PyGObject.
>
> Sounds great. I would like to go through the existing override doc
> strings at some point and update them for better compatibility with
> Sphinx/reST. Would this be helpful or would it be in conflict with
> what you are thinking? (at least the Python only extensions like
> Property and Signal should be well formatted).

Sure, that would be helpful.

> > > * Better developer workflow. By using Python doc strings, we
> > > automatically integrate with all of the awesome Python developer tools
> > > and things like doc tips in IDE's should just work.
> >
> > Is there any IDE/editor which supports some sort of auto completion for
> > PyGObject right now (using the gi.repository import hook)?
>
> I was a bit overzealous with that statement. "should just work" is at
> least true in ipython standalone or integrated with eclipse/pydev
> (which is good enough for me at the moment). Eclipse nor Anjuta work
> for editor auto-completion with import hooks. At a cursory glance,
> Eclipse seems want to import the actual Python modules so there is
> some hope for this working in the editor. Anjuta uses python-rope
> which seems to use AST parsing and would be a more grave situation
> with import hooks. A little more reading of PEP302 reveals there was
> some talk about adding a "list_modules" method to the importer
> protocol which would potentially fix some of these problems, but it
> doesn't seem this has ever been added.

Yeah, a possible solution is to create a fake package containing the whole
API (Google gives me fakegir [0], but I haven't tried it). I think IDE
integration deserves a page somewhere..

> > > * Tools like Sphinx [1] could be used to generate html docs by
> > > pointing it at the "gi" Python package. In a similar vein, I realize
> > > Christoph's pgidocgen uses gir files translated to reStructuredText
> > > which is then run through Sphinx (please correct me if I'm wrong
> > > here).
> >
> > In a limited form "pointing it at gi" should work then. It doesn't handle
> > import hooks afaik, so you would still have to generate rst files with all
> > autodoc references in them. Maybe provide it as a helper in PyGObject
> > similar to sphinx-apidoc.
>
> Same problem as above I guess. Also this approach might not be that
> great if different people try to re-generate the html docs (in the
> context of building docs for hosting). You might get a different set
> of docs from the previous generation due to a different set of
> installed typelib/gir files. In which case managing an explicit list
> of modules we want html docs for might be a way to enforce a
> consistent set of documented modules. Unless of course an automated
> builder is generating the docs based on some standard install like a
> GNOME continuous snapshot.
>
> > > * By using Sphinx, we also get direct references back to the Python
> > > docs [2] for native Python constructs (as is realized with Christoph's
> > > docs, although I'm not sure if it is Sphinx or pgidocgen doing that).
> >
> > This is possible through the Sphinx intersphinx extension [0]. Sphinx
> > creates a "object.inv" [1] file which can be used by other sphinx based
> > projects to resolve references. This is, as you've said, used to link to
> > the official python documentation + the pycairo docs (see Gtk.Widget.draw()
> > [2] for example). It's also used in Sebastian's GTK+3 tutorial to link to
> > the API reference itself.
>
> I didn't realize there was also referencing with pycairo and the
> tutorials on readthedocs, very nice!
>
> > > * Sphinx also seems to supports devhelp output (among other formats)
> > > which is interesting in that we might be able to achieve a similar
> > > look and feel with the rest of the GNOME developer docs.
> >
> > I've tried the devhelp export and it seemed to work quite well for the all
> > in one API docs. But I'm not sure how linking between different Sphinx
> > builds would work with devhelp. (but I have no idea how devhelp does it
> > with other sources either..)
>
> The devhelp output wasn't really what I was expecting, which was
> style consistency of the html output.

Style is fixable, it's just jinja templates. I could try to match the
"official" css, but that's then another thing to maintain and sync of
course. Disabling the sidebar and breadcrumbs should work for starters.

[0] https://github.com/strycore/fakegir