Re: future of scaffold

[Jeroen Zwartepoorte <jeroen xs4all nl>]

Hi John,

On Tue, 2003-12-30 at 06:36, John (J5) Palmieri wrote:
> Ok, here are my comments,
> 
> My biggest worry is that Scaffold is striving to be too pure without
> getting any realy work done.  When Scaffold (Anjuta 2) went through its
> first itteration of changes it lost a lot of functionality.  When the
> merger of gIDE and Anjuta were announced I could actualy use the IDE to
> get work done.  The project manager worked, it would build projects and
> I could click on errors and go to the line that was causing the errors. 
> A little after that some refactoring went on and a couple of things got
> broken, syntax highlighting wouldn't work.  Even now with the complete
> move to GtkSourceview syntax highlighting is a bit sparse.  This is all
> alpha code which is cool but if we go in a direction I just want to be
> able to use what we have and not just rewrite everything.  I happen to
> like the codebase.  It is one of the cleaner ones I have seen in awhile
> but if something useful doesn't come out of it Scaffold will just be a
> neat experiment.  To be sure a lot of nice things have come out of its
> experimentation but I think three name changes later it needs to start
> standing on its own.

I understand your concerns. The reason we're discussing
"yet-another-redesign" is that we encountered problems implementing some
of the more complex stuff in scaffold. A couple of months ago, i was
working on getting a plugin to add a context menuitem to the
project-manager's popup menu in the projectview. This proved to be
everything but straightforward. It meant that a plugin had to use
functionality of another plugin (the project-manager). The
project-manager doesn't have an interface like the document-manager
does. So this kinda start the discussion about how plugins should use &
extend each others functionality.

> With that rant over :-) I think Gustavo's approch makes the most sense. 
> The way I see it is we define a couple of strict interfaces which every
> plugin can rely on. We also have the dynamic interfaces that can be
> descovered by introspection.  If those dynamic interfaces become heavly
> used we move them into the static relm at the next release.  I personaly
> don't think we are going to have to worry about that for some time to
> come.

I agree. Looking forward to the API proposal :)

> As for having to keep bindings upto date I am working on something that
> will allow Scaffold to support its own development.  I am sure you have
> read about my posts on code generation.  Basicly the engine will keep
> tabs on the code so as to have realtime information about classes,
> methods, properties, signals, and interfaces.  I am hoping to hook this
> up to the symbol browser which will allow for such things as clicking on
> a method of a class and telling it to export this method as a
> python/bonobo/perl/your favorite language, binding.  I took this week
> off to work on that and am hoping to have something to show at this
> years NY LinuxWorld.  Of course it will be while before we have the
> ability to export bindings but if we can get Scaffold to a point where
> it can support its own development we will gain more developers and can
> incrementaly refactor where we see the need.

Cool, sounds a bit like your code will allow for refactoring of
GObject-based classes as well?

> More comments bellow:
> 
> On Mon, 2003-12-29 at 22:31, Gustavo M. Giráldez wrote:
> > Hi!
> > 
> > (Warning: long mail)
> > 
> > I want to have as many people involved in this discussion as possible,
> > since that will bring new ideas and points of view to the table.  That's
> > why I've cc'ed the list.  Please, anyone interested in hacking Scaffold,
> > speak up, share your thoughts, so we can build a solid infrastructure
> > for a great Gnome IDE.
> > 
> > On Fri, 2003-12-26 at 11:16, Jeroen Zwartepoorte wrote:
> > > Hey Gustavo,
> > > 
> > > So i'd like to put some thoughts of mine about the future of scaffold
> > > down here.
> > > 
> > > The Mono guys are porting SharpDevelop to Gtk# atm. What they're doing
> > > is changing the SWF (System.Windows.Forms) code to Gtk#. They're not
> > > using gtksourceview. They'll likely use GdlDock in the GUI, but that's
> > > about all it will have in common with Scaffold. So it looks like
> > > Scaffold# is not very attractive to them. (i haven't done any work on it
> > > lately anyway. Working on GdlDock mostly atm)
> > > 
> > > Your comment about making python plugins work in scaffold started me
> > > thinking. I'm thinking we can make a small core, written in C that uses
> > > the new Action API in gtk 2.3 and have plugins around that written in
> > > C/Python/C# etc. This would require some refactoring.
> > >
> > > The reason i'm in favor of something like this is that i'm still not
> > > happy with our current architecture of having a DocumentManager
> > > interface and an implementation of that in NotebookDocumentManager. It
> > > really sucks when you have plugins depending on other plugins etc.
> > > What's your opinion on this? Has it changed after reading the JBuilder
> > > Opentools document? :) (/me hopes it has)
> 
> Plugins should depend on interfaces.  If the interface is not
> implemented the plugin should degrade gracefully somehow.  For instance
> if someone was using a custom text plugin that did not support syntax
> highlighting a plugin that wished to highlight a portion of text should
> simply not use highlighting.  

I disagree here. IMO, a plugin should advertise which plugins is depends
on in its .plugin file. The shell should check that those dependency
plugins are available and can be loaded. If a dependency class is not
available or can't be loaded, the plugin itself shouldn't be loaded
either. If you do it the way you propose, you'll get horrible code to
check whether a plugin is loaded and do something one way and special
code to do it another way when the dependency is not available.

> > 
> > I think using the Action API solves just a small part of the problem,
> > i.e. the UI merging of the different plugins.  It can also be used for
> > plugins offering a visual artifact on which the user can perform
> > actions, such as a file entry in the project view.
> 
> > I agree Scaffold's core should be as small as possible, and have most
> > functional parts implemented as plugins.  I think that's the spirit of
> > the current implementation.
> 
> It would be great if Scaffold's core framework could be used as a
> container for other projects and place anything specific to Scaffold in
> plugins but making Scaffold useful by itself should come before making
> it useful for all.

This is pretty much possible already. You just have to rename some stuff
in scaffold/src to get a working shell with a different name.

> > A single plugin should be also as small as possible, and do one thing
> > well.  The problem is that plugins should interact with each other, to
> > allow the user to accomplish more complicated tasks, and to be able to
> > present more useful information.
> 
> agreed.  I think plugins should be able to interact with each other if
> they can but also be able to do something useful on its own.  For
> example the project viewer should be able to view projects but if the
> editor plugin is present it should be able to open up files in that
> editor.  Perhaps if they are not present it can fallback on mime-type
> and load the appropriate application.

I disagree here. See my reasons in the comments above.

> We could also have a passive interface for interaction where we define
> common signals that any plugin can listen too or emit.  If a plugin
> registeres to listen for that signal it can obtain the data it needs
> without ever having to know about the other plugin.

That's tricky. I much prefer to stick with dynamic interfaces for that.
Adding yet another way for plugins to communicate confuses things imho.

> > Personally, I don't care whether the plugins are written in C, C++,
> > Python, C#, or any other language of choice.  In fact, I'd like to have
> > plugins written in as many languages as possible, and have each one talk
> > to each other in a clean and simple way.  If a plugin implements a
> > service which could be used by other plugins to make the user experience
> > better, it should be possible to export the service and be used from
> > outside without much trouble.  To accomplish that, we need to clearly
> > define the interfaces which plugins will realize or use.
> > 
> > As I see it, we have two alternatives: compile-time interfaces and
> > runtime interfaces.  For compile time interfaces, the plugin needs to
> > know the interface when it's built, and for runtime interfaces, the
> > methods and properties can be discovered during execution, using
> > reflection mechanisms.  Both have advantages and disadvantages, and I
> > think are suitable for different uses.  That's why I'll propose to use
> > both:
> > 
> > a) Have a set of predefined core interfaces (implemented using
> > GInterface) for very basic IDE functionality: a source file (or
> > document), the project, a build target, etc..
> 
> Sound good.
> 
> > b) A runtime system to discover extra functionality, so we don't need to
> > change the core interfaces whenever we need to do communication between
> > two plugins.
> 
> This is also good but I would add when an interface becomes popular it
> is moved into the core (but can still be discovered dynamicly).
> 
> > The other piece of the puzzle, which is a service the shell should
> > provide, is a mechanism to register and retrieve objects which implement
> > the above interfaces.  The current value storage system is a solution to
> > this, but it'll probably need some changes and extensions.  For example,
> > I'd like to have the objects organized in a hierarchical fashion (note
> > that the hierarchy is also a kind of interface).  
> 
> I implemented something like this for my C++ plugin system SOMELib.  It
> basicly kept a catalog of arbitrary attributes that could be used to
> generate a sort of hiearchy (i.e. give me all objects of type poem; from
> that execute print() on all objects with author John).
> 
> An attribute system might work well here.  Give me type=editors with
> status=current.  Give me type="project manager factory". 
> 
> > Another addition I'd
> > like to see is a moniker-like resolution system so any plugin could
> > claim a subpath of the hierarchy, and provide factory and/or directory
> > services.
> > > Is this doable though: if you have plugins written in different
> > > languages, can you still depend on them easily without having to write
> > > bindings for a plugin first? .Net gets around this by having a common
> > > Intermediate Language. We don't have that in C...
> > 
> > Well, we don't have an intermediate language, but we have the GObject
> > runtime system, which I think is good enough for our purposes.  As long
> > as we have gobject bindings and we provide bindings for the core and
> > shell interfaces, we can use almost any language.
> 
> For now I would say this is best.  If we start worrying about other
> languages too much we might end up writting yet another intermediate
> language.

Nod.

> > The idea is not to depend on a particular plugin at compile time, i.e.
> > don't #include a plugin header from other.  If you ever need to do that,
> > that means you should instead use a runtime interface (a gobject
> > property for example), or define a new/extend an existent core
> > interface.
> 
> Exactly, sounds right to me.
> 
> > Ok, so this was all very vague.  My intention is to get the ball rolling
> > and get more people involved in the design.  I'll try to follow up
> > shortly with a more concrete proposal of an API for the shell and a set
> > of core interfaces, but I think I've outlined the basic functionality
> > I'd like to see:
> > 
> > - shell with service/object registering capabilites (to allow plugins to
> > advertise themselves), ui merging and session managment (so plugins can
> > save project independent data)
> 
> I think the session code is good for this already so no work there.

The session code is ok. Might benefit from another pair of eyes though
(it was the first time i wrote such a thing).

> > - a moniker interface, for container-like plugins. E.g. the document
> > manager could claim /Documents and create a new object exporting the
> > document interface, whenever someone tries to access /Documents/foo.c.
> 
> cool.
> 
> > - a UI merging interface for plugin visual elements (even the shell
> > could implement it), to allow plugins to add context actions and/or
> > information to some UI elements from other plugins.  A typical example:
> > a version control plugin would want to add version information and
> > operations (update, commit, etc.) to a source file in the project
> > plugin, which is a row in a tree view widget.
> 
> This interface should be implemented by almost all plugins.  For
> instance I would like the ability through glimmer/gtksourceview to add
> handles (a little dot the mouse cursor can grab) to be able to move
> whole functions without having to resort to copy and paste which often
> gets annoying when dealing with functions that span more than one page. 
> The reason gtksourceview would not implement this is because it has no
> notion of a function block nor should it.  Of corse there can always be
> code in glimmer to gather information from the code parsing plugin and
> provide the functionality if it is present.

Apart from the cool idea, i'm not sure gtksourceview/glimmer will
support that :)

> > - a document interface, offering services such as save, get text, insert
> > text, etc. and notifications of user changes.
> 
> This is one of the most important interfaces.
> 
> > - a project interface, for example to allow other plugins to save
> > project dependent data, enumerate targets and source files, get local
> > configuration information, etc..  Some users of these services: a
> > debugger, a packaging plugin, etc..
> 
> and the code parser to know what code to parse.
> 
> > - a build target interface, with methods such as build, clean, enumerate
> > sources, add source, etc..  Maybe project and target interfaces could be
> > combined.
> 
> They should be seperate interfaces I think, but they may be implemented
> by the same plugin.
> 
> > Thanks for reading this far.  Please comment on this.  Ideas, criticism,
> > corrections, any kind of feedback is welcome (as long as it's
> > constructive ;-)
> 
> You mean no flaming. Oh shucks ;-)
> 
> > Btw Jeroen, the JBuilder Opentools documents did give me some ideas
> > which are part of what I'm proposing.  For anyone interested, here are
> > the URLs:
> > 
> > http://linuxberg.vc-graz.ac.at/mirror/jbuilder/techpubs/jbuilder9/opentools.pdf
> > http://info.borland.com/techpubs/jbuilder/jbuilder9/opentools/ref/OpenToolsAPI/
> > 
> > Regards,
> > Gustavo
> 
> What about the Eclipse way of doing things?  It seems plugins are
> shooting up everywhere you look on that project.

I'm not too familiar with Eclipse, but i usually use JBuilder as a
reference. It was one of the first big IDEs to use a plugin system from
the ground up. Eclipse probably does things in a similar way (as does
SharpDevelop).

> One thing I would like to add.  We need to be more responsive to other
> developers.  There have been people popping up here and there, asking
> questions and submitting patches.  I don't mind answering them but I
> don't always feel authoritive.  People are interested in helping but
> sometimes they get discouraged from the lack of traffic.

Partially my bad. I wasn't really motivated a while ago to work on
scaffold, but i'm starting to get motivated again :)

You mentioned that you're collecting some patches for the debian build?
Could you post a list of the patches you're currently using that haven't
been committed to CVS?

Thanks,

Jeroen