Re: [g-a-devel] ATK - Signal indicating new AtkObject creation.

[Mark Doffman <mark doffman codethink co uk>]

Hi Willie, Michael,

Its just about hanging together thanks Michael. :)

> On Thu, 2009-01-15 at 13:29 -0500, Willie Walker wrote:
>> Thanks for the response, and nice hearing from you.  :-)
> 
> 	Ah - good to shake some of the dust off, (and verily, the ivy that
> groweth over me ;-) [ or something ].
> 
>> If we limit the a11y hierarchy to what is burning phosphorous on the 
>> screen, then we definitely can reduce the number of accessible objects 
>> quite a bit, agreed.  But, I have a lot of questions...  :-)
> 
> 	Ah yes - so, if we do that we also suck quite a lot ;-) I propose
> instead some "near the screen" - metric. ie. anything that is a limited
> set and has a well bounded lifetime should be immediately at hand, and
> consistent in the AT's local tree representation.
> 
>> I'm scratching my head about things such as NODE_CHILD_OF, 
>> LABEL_FOR/LABELED_BY, CONTROLLER_FOR/CONTROLLED_BY, FLOWS_FROM/FLOWS_TO, 
>> etc., relations where the related accessible is not on the screen.
> 
> 	Yep - these are great cases with the 'near-to' stuff - I'd also like to
> have all menus, and toolbar items exposed: regardless of whether they
> are popped up and down, and of course, partial occlusion of a window
> should have no bearing on what is exposed :-)

The way I think about it is that an accessible object is just an adapter
exposing the information of another object, a Widget or dom-node or
something else. If that object is in-memory then its accessible adapter
should be available.

This might not make sense for very small objects, where the adapter
might be larger than the objects themselves. In that case Michaels
phrase 'anything that is a limited set and has a well bounded lifetime'
might make more sense.

> 
>> Or, imagine we have a large list that is a few items larger than the 
>> screen.  Does the child count of the large list only include the objects 
>> on the screen (that would be odd) ?
> 
> 	So - this is basically a variant of the MANAGES_DESCENDANTS thing - if
> the list is staggeringly big, we need to have a different behaviour from
> if it is say 20 elements: if the latter, I'd advocate exposing them all,
> if the former even to expose them to a non-impaired person presents
> challenges.

For a very large table list or document; something where many of the
objects in the list may not be in memory, we need a way to navigate the
list or document. One way (That sucks) would be to only create and
expose the objects that are on-screen. So for a very large document the
AT would have to twiddle the 'Value' interface on the scrollbar to read
down the entire document.

An off the cuff interface:

Accessible object implements the "Table" interface it has the
"manages-descendants" state. It also implements the imaginary interface
"TableRange".

The "TableRange" interface has the following method:

Range registerClient ()

The Range interface has the following method:

setRange (minRow, maxRow, minColumn, maxColumn)

The setRange object would do as imagined, create and make accessible all
the objects within a certain range. The purpose of the registerClient
call is that there is one Range object per-client / AT.

The advantages of this interface over reference counting in the infinite
space is that there is no chance of creating a memory leak in the
application via poor reference counting in the AT. There are far fewer
IPC calls to constantly ref and unref objects in the server/application.

>> I'd guess maybe what you're proposing is that the app always dumps the 
>> a11y hierarchy for only the stuff that's burning phosphorous, and that 
>> references/queries to stuff off screen ends up being created lazily.
>> Seems like it might be workable.
> 
> 	Nah - lets dump everything on, or 'near' the phosphorous to the AT.
> 
>> In this new world order, however, will an AT still be able to hang onto 
>> the handle of an accessible for the purposes of identity/equality 
>> comparison across events?  An example use case is where a screen reader 
>> wants to gather information about where focus used to be and where it is 
>> now.  It does so by listening for focus events and saving the event 
>> source away.
> 
> 	Sure - of course, while the object that got focus and retained it is
> still near/visible. However, the lifecycle of that object will be clear
> - when it is not useful [ie. destroyed ], it will be gone. Currently I
> believe we can save an accessible handle away for focus handling with a
> ref count for an object that is long destroyed and defunct - and this
> will retain state in the application itself ;-) Of course, AT's should
> still be able to keep & compare the handle in future, but there will
> just be no app-side resource associated.

Pretty much what Michael said. Pyatspi as written makes comparisons
based on the D-Bus object path (Handle). So even when the implementing
object has been destroyed its handle will remain around for comparison
purposes.

> 
>> One obvious use case related to this is a "Say All" feature for a screen 
>> reader such as Orca.  In this mode, the user presses a key and Orca 
>> starts reading the document until Orca reaches the end of the document 
>> or the user presses a key to stop the operation.  While doing this, Orca 
>> may also highlight the words as they are being spoken.

Well something like the TableRange interface or DocumentRange in the
same vein may work. This would mean updating the range as the document
is read.

> 
> 	Yep - we need an API like that anyway IMHO.
> 
>> The API needs to be able to provide Orca with the ability to get the 
>> information in the document in sequence, preferably by grammatical 
>> structure (e.g., sentence by sentence, paragraph by paragraph, etc.), 
>> and also make sure the text is visible on the screen.

Add an iterator-like API to the DocumentRange interface. Call "Next" to
make the next 5-6 paragraphs available. This needs some thinking about,
 my idea is that the AT will only get updated about new objects at the
Idle loop, meaning when "Next" is called the next paragraph Accessible
object would not be instantly available.

> 
> 	Sure; sounds good to me.
> 
>> Also related to this would be what we would need to do in cases where a 
>> large text object is partially on/off the screen.  Gedit, for example, 
>> has a single text object that visually shows a window of what could be a 
>> very large text document.  What happens with the text that is not on the 
>> screen?  Does all of its accessible text get sent along with the hierarchy?
> 

Currently the text doesn't get sent with the object.

> 	Again - I'd say near screen: whatever makes sense here. Wrt. sending
> all of the accessible text - that's an open question: clearly for static
> items, it makes a lot of sense [ IMHO ] to have as much data instantly
> accessible to the AT as possible, without any round-trips.
> 
>> I believe the Collections API is supposed to help with this, but I think 
>> it has some implementation issues that can cause it to actually behave 
>> slower in some cases.  :-(
> 
> 	;-)
> 
>> It's definitely good to have these discussions.  I was mostly on the 
>> provider side of things (i.e., the infrastructure side) for many years 
>> prior to working on Orca.  I've now been on the consumer side for 
>> several years.  Life is much different on the consumer side.  :-)
> 
> 	Sure - and I love your feedback :-)
> 


Thanks everyone,

I really appreciate the interest.

Mark