Re: Touchscreen Compatibility [was: Feature proposal: combined system status menu]

[Tristan Van Berkom <tvb gnome org>]

I've been holding back from participating in this discussion, because I
don't really want to cause unrest, but I feel I should say some things
from the POV of someone who has been working with touch interfaces
for over a decade.

First, I think it's a noble effort to try to provide a desktop environment
which is both touch friendly and mouse friendly, but we should really
be careful and recognize the challenge it presents.

Also, we should make no false illusions that this effort can easily carry
through to applications which run in this desktop environment, applications
from system settings configuration UIs, text editors to audio/video player
applications, all need special care when adapting to touch.

There are two paths I can see which an application can take to be
touch friendly:

a.) Distribute a completely separate binary designed for touch

b.) Try to do what gnome-shell seems to be trying, i.e. detect
     whether the app is running in a touch environment, and
     make specific tweaks, use separate UIs for the touch environment
     than the app would use with a mouse driven environment.

On Tue, Apr 30, 2013 at 6:37 PM, Allan Day <allanpday gmail com> wrote:
> Juanjo Marín <juanjomarin96 yahoo es> wrote:
> ...
> > But also some problems has arisen in the effort of being compatible with touch devices.For example, I 
> > think that the UI of new applications like Documents are very touch friendly, but it's weird for keyboard 
> > + mouse users. It is weird because the interaction is very different from other core applications like 
> > Nautilus (Files). In Nautilus, double click opens a file, but only one click opens it in Documents, and 
> > the way of selecting elements and doing actions with selected elements is quite different. I think 
> > Documents works great in touch screen devices and it is a little bit clumpsy with mouse, and Nautilus 
> > works great in mouse and keyboard but not so good for touchscreen devices.
>
> Right, so the selection design pattern is probably the most prominent
> place where touch compatibility has had an impact. It should be
> emphasised that it is somewhat unique in this regard.

This comment is why I was prompted to jump in to this conversation
actually, I can point out a couple examples where apps in Touch
environments should behave differently, asides from single/double
click / kinetic scrolling and behaviour of input events.

a.) Larger buttons, larger screen area to catch events.

  This has already been mentioned before, and is probably
  the least significant issue (applications can provide larger
  buttons in Touch environments probably easily, without
  changing the UI functionality much).

  However, it should not be ignored that Touch requires very large
  screen areas compared with Mouse driven environments, and
  in Mouse driven environments you really, really want to economize
  screen realestate by making some controls small.

  An example of this would be the toolbar in LibreOffice, this simply
  cannot be done in a Touch environment realistically without really
  offering a separate UI for Touch envirnoments.

b.) Behaviour of sliders, aka GtkScale

  This one has been overlooked I think, and is very important
  to note.

  On our Touchscreen operated Jukebox system we had custom
  widgets for GtkScale to deal with the different expectations one
  would have from a Touch driven environment.

  The slider had 3 different modes to cope with sliders used in
  different scenarios:

    1.) Browser Mode

    Browser mode is used when navigating in a list of content,
    for instance a window full of album jackets with a slider,
    where the current letter index into the content is printed
    on the "knob"

    The specifics of Browser mode is that the "knob"/"position"
    jumps directly to the click position whenever the "trough"
    area receives a button press event.

    In this case, you really dont want the behaviour of GtkScrollBar,
    which jumps "page size" in a given direction for every click
    on the "trough" area, instead you want the knob to go directly
    to letter "M" or whatnot.

    2.) Volume Mode

    When interacting with a volume control "knob" on a slider, I
    cannot stress enough how paramount the importance is to
    not have "Browser Mode" functionality in Touch.

    I.e. when playing with the master volume (or "Bass" control
    slider even), you never want to risk that the volume jumps
    to the maximum when touching the "trough" area above the
    "knob".

    In this case, what you want to do is let the finger press
    the "trough" area, and not modify the slider position *until*
    the finger moves into the "knob" area... once the finger has
    entered the "knob" area, then you want the knob to start
    following the finger position until either the finger leaves
    the slider area completely or there is a "button-release-event".

    3.) Strict Mode

    This is like the above mode I described as "Volume Mode",
    except that it *never* modifies the "knob" position at all unless
    the initial "Touch" was actually made inside the "knob" area itself.

    I recall we had this on the Jukebox but I can't recall why, the
    above 2 modes are most important to distinguish between in
    Touch environments.

In any case, I think it's commendable to attempt to provide some
apps which behave equally well in pointer driven environments
as they do in touch environments, but it would be a mistake to
think that it could be done with exactly the same code and exactly
the same UI/layout/widgetry. Some special cases need to be made
by the actual applications in order to perform well in both
environments.

Best Regards,
        -Tristan

> The selection pattern has been evolving a bit, and we have a round of
> design changes planned which we will hopefully happen this cycle. Me
> and Jakub literally have a list of things that can be done to the
> selection mode to make it better with a pointer. Once we're done I
> don't think it will be any worse than the selection mechanisms that we
> have in nautilus today.
>
> It should also be said that this pattern does have benefits when you
> are using a pointer. An obvious example of this is the difference
> between single/double click. Not only is double click not exactly
> ideal on a touchpad, but it is also used inconsistently and is
> non-discoverable (some places you need double click to open, others
> you need single.) In general, using single click consistently is a
> much better approach, especially when combined with discoverable
> mechanisms for selection.
>
> Allan
> _______________________________________________
> desktop-devel-list mailing list
> desktop-devel-list gnome org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list