next steps for touch support in GTK+

[Simon Schampijer <simon schampijer de>]

Hi,

there have been several discussions around GTK+ touch support during 
GUADEC. To get a good touch experience we talked about the following 
points. I ordered them by priority that we set in Sugar development for 
them. If you come across something that we missed, please comment.

=== 1. Text selection ===
A lot of work has been going into text selection and manipulation those 
days from the design and implementation point of view. The current 
design ideas can be seen at [1].

Carlos created universally usable handles (implemented using GtkWindows) 
to adjust the selection and wrote the code to use them in GtkEntry and 
the GtkTreeView in this branch [2]. The idea is to make those available 
as well for external widgets, e.g. in Abiword or WebKitGTK.

We discussed as well how the Edit Palette should look like and agreed 
loosely on using GtkWindows instead of an Overlay for this to handle 
cases like resizing windows for example.

Something to check is if we want to allow to adjust the selection in 
moving the right handle over the left one. Both iOS and Android do block 
that and snap to the minimum selection when releasing the finger which 
seems a good interaction model and limits complexity and edge cases.

Another thing to make sure is that we do not use fixed positions in 
regards to Wayland.


=== 2. OSK - Slide canvas vertically to keep insertion point visible ===
Carlos has been working on this already. He will send a mail with more 
info about it. To get more background about the issue itself a 
whitepaper is available at [3].


=== 3. SpinButton ===
We had several discussions around the SpinButton. One thing that came up 
is why we do not want to reuse the existing one. One option is to put 
the '<' on the left side (this could be done by a property for example). 
When the SpinButton was redesigned they put the < and > close together 
to not interfere with other widgets at the left side. Having something 
like "[button] < | 3| >" might not be as clear. That at last was the 
reasoning for putting the stepper buttons next to each other. In touch 
the interaction gets a bit fiddly having the buttons next to each other 
that is why we propose that configuration option.

We would keep the entry widget, on touch you can use the OSK to set a 
value, or you can set it to be non-editable. Furthermore you can accept 
a vertical swipe to alter the value, implementing this would be a 
perfect use case for EventControllers.

Another option is introducing a complete new widget targeted at touch 
usage (similar to the one in iOS Garageband) [4] which Carlos 
implemented already [5]. The issue is here the height, which at least in 
a Sugar toolbar could be not as ideal and introducing a new widget.


=== 4. Which GTK+ widgets break with touch ===
The SpinButton item from above is one example of those. Another one was 
the Combobox which has been fixed by Carlos and landed in master [6]. I 
will have a look if I come across more. Feel free to add if you are 
aware of more blockers.


=== 5. Widget Gestures (e.g. swipe, pinch) ===
The desired implementation is to use EventControllers for that. The work 
has currently stagnated on that end. In Sugar we could implement this at 
our toolkit level but an upstream solution would be desired at least 
longer term.


=== 6. OSK widget context provider (e.g. search vs open vs go...) ===
Matthias said there was a patch floating around for that. I looked in 
the bugs with patches attached in bugzilla but could not find it. If 
someone knows where it is would be great.

Regards,
   Simon

[1] https://live.gnome.org/GnomeOS/Design/Whiteboards/Selections
[2] http://git.gnome.org/browse/gtk+/log/?h=touch-text-selection
[3] 
https://wiki.maliit.org/images/a/a7/Technical-overview-widget-relocation.pdf 
(from https://wiki.maliit.org/Documentation#Technical_documentation)
[4] http://wiki.sugarlabs.org/go/File:Ios_spinbutton.JPG
[5] https://bugzilla.gnome.org/show_bug.cgi?id=678108
[6] https://bugzilla.gnome.org/show_bug.cgi?id=678113