Re: [g-a-devel] Questions about the gnome-mag design



Hi Carlos:

By the way, I did not mean to sound too negative in my reply. Your questions are good ones, and I really thank you for your continued interest in gnome-mag.

I agree with you that the separation of these two process are important and I think that I really understand the importance of CORBA (or other IPC mechanism) to achieve this goal.

The desire to have the magnification "decoupled" from the client processing means we need either IPC or multi-threading. My impression is that in a language like C, multi-threading is probably even harder to get right than IPC.

Also, there are situations where the magnifier service may need to run on a different host from the client. For instance, if you are running a remote application, you don't really want the magnifier to run remotely as well, because of the much larger amount of X traffic and the CPU requirements of the magnification service. CORBA allows this at least in theory (although our current 'bonobo-activation' mechanism does not recognize remote servers). This sort of case is probably more common in some corporate environments, but we are not sure just how important it will be in the future.

Just for information, what is scrolling?

Sometimes I used the words "scrolling" and "panning" interchangeably. The two primary tasks of a magnifier, I think, are creating a modified (rescaled) copy of a source image and keeping it up to date, and moving that rescaled image around on the target display, to keep it in the "right" position for the user. The second task, moving a large image around with respect to a smaller viewport, is what I call "panning" or "scrolling".

It seems that from the user's point of view, the most important thing is that this scrolling happens as rapidly and smoothly as possible. If the scrolling is jerky or delayed, the user perceives that there is a performance problem with the magnification. However, the user may be able to accept a delay of a few milliseconds in actually "updating" a region that has changed since it was last visible inside the viewport. This is why we separated the scrolling from the update processing inside gnome-mag, with different priorities. This is also one of the complexities that can make the existing gnome-mag implementation code hard to understand and modify.

Technical note: On some other platforms this scrolling can be done faster and more smoothly for a number of technical reasons. But the XWindow system doesn't really have a "scroll" command that we can use directly, all our scrolling has to be accomplished with XCopyArea and the generation of "expose" events. Most of our scrolling requires that data be copied from the gnome-mag process memory to the XServer for all the newly-exposed areas, and we don't have a way to do this during the vertical blanking interval. These are just some of the technical challenges facing the magnifier. Some of the are made better with the COMPOSITE extension, and a magnifier based on the OpenGL APIs might also have an advantage, since it would probably be able to take advantage of hardware acceleration at least on some systems.

What we do now is "scroll" the existing client-side rescaled image into the viewport quickly, then process any resulting "updates" that need to happen in order to keep the rescaled image up-to-date with the source image. We use the DAMAGE extension to get notification of when the source image has changed, and the XFIXES extension to get information about the onscreen mouse pointer so that we can rescale and re-display that as appropriate.

...


What do you think in an effort to merge what BAUM developed in gnopernicus with gnome-mag to create a complete solution for magnification, eliminating these CORBA dependecies/complexities?



This was explicitly avoided when gnopernicus, gnome-mag, and at-spi were initially designed, for many reasons.


Could you list the more important/relevants reasons? They are these ones that are still on the text, or have any others?

One reason is that gnopernicus is not, we hope, the only assistive technology to be developed for the free desktop, and we also hope that the existing 'sample implementation' of gnome-mag is not the only magnification service that will be written. Already we have the 'orca' screen reader which probably will support magnification someday, but perhaps someone else will write a different client? Some users want speech and magnification together, but some users may benefit from a smaller "magnification only" client. Also, like orca, ATs might be written in languages other than C, which is another reason for using IDL because multiple language bindings are defined by the OMG standard.

So the existing design allows multiple clients to use the magnifier, separately or even at the same time, and it also allows clients like gnopernicus to choose which "gnome-mag:1.0" implementation to use. For instance there are several ongoing efforts to write improved magnification services (for instance Gunnar Schmidt's is working on a COMPOSITE-based magnifier for school, and I believe there is at least one other effort to use COMPOSITE as a magnifier, and there is Keith Packard's experimental program called 'diopter'). If these programs were provided with gnome-mag IDL wrappers (basically just connecting the 'impl' methods in the CORBA "skeletons"), then gnopernicus could choose to try different versions and select the best service for a given platform and user need. The use of a common interface means that moving from one implementation to another is very straightforward.


Why not design gnome-mag to be a library (or only a program like I said in the last paragraph) that support magnification and export an API in C? I have doubts if making this program an object that can be accessed throw CORBA is the best solution.



Creating a new set of client bindings with a C api may be reasonable, but there is already a C API - the one defined by the CORBA and IDL specification. In any case this would need to wrap a client-server model, so some other IPC mechanism would have to be substituted for the CORBA one. There are a number of areas where magnification can be improved, but I don't think that this one would give much useful improvement compared to the effort involved, and could be harmful.
Better improvements would include:

* a multi-pass rendering system, so that better smoothing algorithms could be added to gnome-mag without slowing the panning performance unacceptably;


Could you explain better what is a multi-pass rendering system or point to some resources?

I described how we already have a sort of "multi pass" system, but it only makes two 'passes', it scrolls the existing scaled image into view, then updates any "stale" regions at idle time. To make better use of smoothing algorithms, a more sophisticated 'multi pass' system would make more than one "pass" when rendering the updated image. The first pass would be a fast, but possibly not high-quality, scaling algorithm like the existing GdkPixmap scaling operations we use. Then, once all the pending "first pass" updates are done, a second pass at lower priority could begin, re-rendering the updated regions with a slower, higher-quality smoothing algorithm. Some text smoothing algorithms are considerably slower than pixel replication or bilinear interpolation, but give less "blurry" results, so this would provide the magnification user with "quick feedback" when moving the mouse ("panning"), and "best quality" when looking at a static image, for instance when reading text.

The idea is similar to the idea behind JPEG images in a web browser that first display a blocky image, and then render the higher-quality image as the rest of the image data is downloaded.

best regards

Bill



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]