Re: Feature proposal: Alternative input system based on low-cost webcam

[meg ford <meg387 gmail com>]

Hi,

I think that this raises an important point about inclusion of
accessibility features: they are often very innovative, and they enhance
the experiences of average users. For example, the on-screen keyboard
can be adapted  for use on  touch-screen devices (at least this is my
understanding from discussions I had with Nohemi Fernandez while we were
in Montreal) - as long as someone extends the code :)

I agree with that, but we should always keep in mind who is the primary target, because sometimes people with disabilities need specific features which people without disabilities don't (imagine, for instance, a filter for the on-screen keyboard to remove unwanted keystrokes due to poor hand control).

I completely agree with you here. I think that it is crucial that GNOME have a culture that supports and understands Universal design. At the same time, I also think that our accessibility community needs to put some more effort into talking to our community about access, and in communicating with the community about how to best integrate features so that they do not interfere with the average user's experience.

Since the trends right now are moving away from mouse use, it might be
helpful/interesting to ask ourselves whether or how cameras can be used
to replace them. Are there situations in which a camera would be more
useful than a touch screen or track pad? Certainly it can be for certain
users with disabilities, but we might want to explore other use cases,
as well.

IMHO (I'm not an expert) we are still in the stone age of the machine vision based interaction and we will see great innovations in years to come. So, sure, there is much to explore and have fun!

BTW: if someone is interested in playing with this, she might find interesting/useful another project called SITPLUS [1] in which we try to find other uses of camera based interaction for people with (severe) disabilities.

[1] http://sitplus.appctarragona.org

    > I -think- that for this to work properly we'd need a bunch of things:
    > first, we need to track not only head movement but also your eyes and
    > several facial muscles so that we can have accurate tracking and
   hints
    > about your

   AFAIK, eViacam developers plans to add support for more facial gestures
   in the future.

Yes. Some users requested to detect facial gestures [2] and there had been some efforts regarding blinking detection [3]. We hope to have time (and resources) to add/improve them soon.

[2] https://sourceforge.net/tracker/?func=detail&aid=2883828&group_id=248049&atid=1199431

[3] http://eviacam.git.sourceforge.net/git/gitweb.cgi?p=eviacam/eviacam;a=shortlog;h=refs/heads/blink_detect

    > intentions. Well, this requires cameras with resolutions much higher
    > than VGA, which is the current standard for these. Then, someone
   needs
    > to figure out how to track all these elements real-time with little
    > cpu usage. As it is, we

   With higher resolution cameras the behaviour would be better. But
   please, read again the feature proposal name "Alternative input based
   system based on *low-cost* webcam".

   About performance, it is something that was always one of the priorities
   for eViacam developers, and the reason that the configuration wizard
   allows you to tweak so many parameters.

    > can't even maintain a Mexican hat over ones head in Cheese without it
    > lagging 3 seconds behind. And finally for this to work we'de need
    > pretty good AI to be able to understand what you really want so that
    > you don't end up sending a draft-mail just because you glanced at
   that
    > gorgeous girl that just passed in front of you.

   See my previous comment about performance.

Performance has always been a primary goal. In fact, is not a coincidence that we chose C/C++, used multiple threads and wrote custom code for the camera capture. And, in my experience, there is no noticeable delay in the capture - control loop is the camera is properly set up. In fact, I still remember running earlier versions on a PIII 450Hz laptop without problems.

I  also think that performance and facial gesture support will most
likely improve faster if this is included and available to more users
and developers.

Sure.

Best wishes,
Meg