Re: New interface: AtkStreamableContent

[Peter Korn <peter korn sun com>]

Hi Bill,

I think this is an excellent idea - having a clean, formal interface to get at
underlying the content used for various pieces of the user-interface.  We have
gotten significant feedback from assistive technology vendors that they want to
not be limited by the accessibility API, but be able to go directly to data that
they know how to parse, and present that data in their own way.  The most common
place this was cited was with HTML, but I think we should look at this request
in a general way.  A generic "streaming content" interface, with mime type and
raw data, may be a very nice way to do this.  My only real concern is the
semantics of interacting with this interface.  Should we have a simple stream
interface?  Should we buffer the data, and allow any kind of more random access
to the data? (I think the answer here is no).  How do we define multiple
different mime types available for the data?


Regards,

Peter Korn
Sun Accessibility team


Bill Haneman wrote:
> 
> Partly as a result of conversations about the utility of AtkImage (and
> the corresponding AccessibleImage interface that would be part of the
> SPI), I have a proposal for a new interface.
> 
> With respect to AtkImage, I think that so far the only reasons we have
> for keeping it are the ability to provide image geometry for images
> without corresponding widgets (a case which we don't have hard
> examples of yet), and the semantic value of saying "I'm an image".
> 
> There was some support for a call like the current
> atk_image_get_storage_type(...) but my understanding was that this
> would be of little use if the image data/pixels were unavailable to
> the AT.  Perhaps this is a noticeable lack in the current Java
> Accessibility API.
> 
> An important use case might be that of a FAX viewer, in which the
> image view is really a CCIT TIFF file (or other FAX format).  In order
> to make this "text" readable, one requires OCR.  I think that it makes
> far more sense for the OCR to live in the AT or be used by the AT than
> to link OCR into the toolkit-level accessibility support (in libgail,
> for instance, or the "Gnome Bridge" to the SPI).
> 
> Thus it would be good for the image to be able to export its mimetype
> and backing data stream to the AT.
> 
> It occurs to me that this is of general interest, not just to images,
> and that in fact it may not always make sense to expose such data for
> images that don't originate as PNG or other bytestreams.  Thus the
> functionality belongs not in AccessibleImage, but elsewhere.
> 
> I don't suggest that accessible objects should do conversion or export
> multiple formats right away - rather that those accessible
> widgets/entities that *do* have backing data streams be able to expose
> them cleanly to the AT.  Thus a GtkHTML widget could implement
> AtkStreamableContent with a mimetype of "text/html" (optionally it
> could export "text/plain" as well but the first mimetype would be
> sufficient).
> 
> The corresponding ATK interface might consist of two methods:
> 
> gchar*[]     atk_streamable_content_get_mime_types
> (AtkStreamableContent *streamable);
> 
> GIOChannel   atk_streamable_content_get_stream (AtkStreamableContent
> *streamable,
>                                                 gchar *mimetype);
> 
> [Please note that I am not sure of the most Glib-friendly way of
> exposing an array of strings or an IO stream is, comments welcome].
> 
> I believe that the lack of exposure of backing data is a significant
> omission in our current architecture (and of other accessibility
> architectures), this would address that need.  The concept could be
> extended, even, to non-visual content types like audio, though it is
> not clear, in a given audio application, where the implementing
> AtkObject would belong in the application's accessible object
> hierarchy.
> 
> Regards,
> 
> Bill
> 
> --------------
> Bill Haneman
> Gnome Accessibility / Batik SVG Toolkit
> Sun Microsystems Ireland
> 
> _______________________________________________
> gnome-accessibility-list mailing list
> gnome-accessibility-list gnome org
> http://mail.gnome.org/mailman/listinfo/gnome-accessibility-list