[gtk+/multitouch: 123/124] docs: Add section about multitouch and other interaction patterns
- From: Carlos Garnacho <carlosg src gnome org>
- To: commits-list gnome org
- Cc:
- Subject: [gtk+/multitouch: 123/124] docs: Add section about multitouch and other interaction patterns
- Date: Tue, 24 Jan 2012 11:35:40 +0000 (UTC)
commit eb44fcf4bda5557587da000c13ac803d5432891f
Author: Carlos Garnacho <carlosg gnome org>
Date: Mon Jan 16 19:08:40 2012 +0100
docs: Add section about multitouch and other interaction patterns
This is a ripoff of http://live.gnome.org/GTK%2B/DeviceInteractionPatterns,
with better phrasing in general, some more factual points, and multitouch
explained.
docs/reference/gtk/Makefile.am | 2 +
docs/reference/gtk/device-interaction-patterns.xml | 578 ++++++++++++++++++++
docs/reference/gtk/gtk-docs.sgml | 5 +-
3 files changed, 583 insertions(+), 2 deletions(-)
---
diff --git a/docs/reference/gtk/Makefile.am b/docs/reference/gtk/Makefile.am
index 54f3c7a..1fc9cf7 100644
--- a/docs/reference/gtk/Makefile.am
+++ b/docs/reference/gtk/Makefile.am
@@ -119,6 +119,7 @@ content_files = \
running.sgml \
building.sgml \
compiling.sgml \
+ device-interaction-patterns.xml \
drawing-model.xml \
glossary.xml \
migrating-2to3.xml \
@@ -142,6 +143,7 @@ content_files = \
overview.xml
expand_content_files = \
+ device-interaction-patterns.xml \
drawing-model.xml \
getting_started.xml \
glossary.xml \
diff --git a/docs/reference/gtk/device-interaction-patterns.xml b/docs/reference/gtk/device-interaction-patterns.xml
new file mode 100644
index 0000000..0e028c3
--- /dev/null
+++ b/docs/reference/gtk/device-interaction-patterns.xml
@@ -0,0 +1,578 @@
+<?xml version="1.0"?>
+<!DOCTYPE section PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
+ "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd"; [
+]>
+<chapter id="gtk-device-interaction-patterns">
+ <title>Multitouch and other device interaction patterns</title>
+
+ <para>
+ Depending on the platform, GTK+ is able to handle a wide range of input
+ devices. Those are offered to applications in a 2-level hierarchy, with
+ virtual devices (or master devices) representing the visual cursors
+ displayed in the screen, which are each controlled by a number of physical
+ devices (or slave devices). Those devices can respectively be retrieved
+ from an input event with gdk_event_get_device() and
+ gdk_event_get_source_device().
+ </para>
+
+ <para>
+ In X11, GTK+ uses XInput2 for input events, which caters for a fully dynamic
+ device hierarchy, and support for multiple virtual pointer/keyboard pairs.
+ </para>
+
+ <example>
+ <title>Listing and modifying the device hierarchy</title>
+ <programlisting>
+ carlos sacarino:~$ xinput list
+ â Virtual core pointer id=2 [master pointer (3)]
+ â â Virtual core XTEST pointer id=4 [slave pointer (2)]
+ â â Wacom ISDv4 E6 Pen stylus id=10 [slave pointer (2)]
+ â â Wacom ISDv4 E6 Finger touch id=11 [slave pointer (2)]
+ â â SynPS/2 Synaptics TouchPad id=13 [slave pointer (2)]
+ â â TPPS/2 IBM TrackPoint id=14 [slave pointer (2)]
+ â â Wacom ISDv4 E6 Pen eraser id=16 [slave pointer (2)]
+ â Virtual core keyboard id=3 [master keyboard (2)]
+ â Virtual core XTEST keyboard id=5 [slave keyboard (3)]
+ â Power Button id=6 [slave keyboard (3)]
+ â Video Bus id=7 [slave keyboard (3)]
+ â Sleep Button id=8 [slave keyboard (3)]
+ â Integrated Camera id=9 [slave keyboard (3)]
+ â AT Translated Set 2 keyboard id=12 [slave keyboard (3)]
+ â ThinkPad Extra Buttons id=15 [slave keyboard (3)]
+
+ carlos sacarino:~$ xinput create-master eek
+ carlos sacarino:~$ xinput list
+ ...
+ â eek pointer id=17 [master pointer (18)]
+ â â eek XTEST pointer id=19 [slave pointer (17)]
+ â eek keyboard id=18 [master keyboard (17)]
+ â eek XTEST keyboard id=20 [slave keyboard (18)]
+
+ carlos sacarino:~$ xinput reattach 10 17
+ carlos sacarino:~$ xinput list
+ ...
+ â eek pointer id=17 [master pointer (18)]
+ â â Wacom ISDv4 E6 Pen stylus id=10 [slave pointer (17)]
+ â â eek XTEST pointer id=19 [slave pointer (17)]
+ â eek keyboard id=18 [master keyboard (17)]
+ â eek XTEST keyboard id=20 [slave keyboard (18)]
+ </programlisting>
+ </example>
+
+ <para>
+ Anytime a virtual device is added or removed, or a physical device
+ is attached to another virtual device, or left floating (detached
+ from any virtual device), #GdkDeviceManager will emit the corresponding
+ #GdkDeviceManager::device-added, #GdkDeviceManager::device-removed, or
+ #GdkDeviceManager::device-changed signals.
+ </para>
+
+ <section id="gtk-device-patterns-client-pointer">
+ <title>The client pointer</title>
+
+ <para>
+ In X11, Under the presence of multiple virtual pointers, GDK and XInput2
+ use the "client pointer" principle to allow several legacy applications
+ to interact simultaneously with different virtual pointer/keyboard pairs,
+ it would be usually set by the window manager for a focused window, so
+ different application windows could operate on different client pointers.
+ gdk_device_manager_get_client_pointer() may be called to get the client
+ pointer #GdkDevice
+ </para>
+
+ <para>
+ Under the hood, X11 uses the client pointer (or its paired keyboard) to
+ satisfy core calls such as XGrabPointer/Keyboard, XQueryPointer and
+ others that have been superseded by XInput2.
+ </para>
+
+ <para>
+ In platforms without multidevice features, gdk_device_manager_get_client_pointer()
+ will return the only virtual pointer available.
+ </para>
+ </section>
+
+ <section id="gtk-device-patterns-simple">
+ <title>Simple device handling</title>
+
+ <para>
+ There are applications that could have little gain in knowing about
+ multiple devices, although there are situations where a device could
+ be needed (i.e. popping up a menu on the pointer coordinates).
+ </para>
+
+ <para>
+ For such applications, the client pointer may be a good enough
+ approximation for these operations. Under the presence of multiple
+ device pairs, this gives a behavior that is most similar to that
+ of legacy applications (i.e. gtk+2).
+ </para>
+
+ <example>
+ <title>Getting the client pointer and keyboard</title>
+ <programlisting>
+ GdkDisplay *display;
+ GdkDeviceManager *device_manager;
+ GdkDevice *client_pointer, client_keyboard;
+
+ display = gdk_display_get_default ();
+ device_manager = gdk_display_get_device_manager (display);
+ client_pointer = gdk_device_manager_get_client_pointer (device_manager);
+
+ /* Or if we need a keyboard too */
+ client_keyboard = gdk_device_get_associated_device (client_pointer);
+ </programlisting>
+ </example>
+ </section>
+
+ <section id="gtk-device-patterns-multiple-devices">
+ <title>Dealing with multiple devices</title>
+
+ <para>
+ There may be several usecases to deal with multiple devices, including,
+ but not limited to:
+ </para>
+
+ <orderedlist>
+ <listitem>
+ Retrieving advanced information from an input device: i.e. stylus pressure/tilt
+ in drawing applications.
+ </listitem>
+ <listitem>
+ Receiving events from a dedicated input device: i.e. joysticks in games.
+ </listitem>
+ <listitem>
+ Collaborative interfaces, handling simultaneous input from multiple users.
+ </listitem>
+ </orderedlist>
+
+ <para>
+ However, the patterns to make them work are very similar.
+ </para>
+
+ <section>
+ <title>Event handling</title>
+
+ <para>
+ Each device will emit its own event stream, this means
+ that you will need to check the GdkEvent you get in
+ your event handlers
+ </para>
+
+ <example>
+ <title>Reacting differently to devices</title>
+ <programlisting>
+ static gboolean
+ my_widget_motion_notify (GtkWidget *widget,
+ GdkEventMotion *event)
+ {
+ GdkDevice *device, *source_device;
+
+ device = gdk_event_get_device ((GdkEvent *) event);
+ source_device = gdk_event_get_source_device ((GdkEvent *) event);
+
+ g_print ("Motion event by '%s', coming from HW device '%s'\n",
+ gdk_device_get_name (device),
+ gdk_device_get_name (source_device));
+
+ /* Handle touch devices differently */
+ if (gdk_device_get_source (source_device) == GDK_SOURCE_TOUCH)
+ {
+ ...
+ }
+ else
+ {
+ ...
+ }
+ return TRUE;
+ }
+ </programlisting>
+ </example>
+
+ <para>
+ The mechanism above could also be used for fine grained event discarding
+ (i.e. so rubberband selection doesn't jump to another pointer entering
+ the widget for example)
+ </para>
+
+ <example>
+ <title>Reacting differently to devices</title>
+ <programlisting>
+ static gboolean
+ my_widget_button_press (Gtkwidget *widget,
+ GdkEventButton *event)
+ {
+ GET_PRIV(widget)->current_pointer = gdk_event_get_device ((GdkEvent *) event);
+ ...
+ }
+
+ static gboolean
+ my_widget_button_release (Gtkwidget *widget,
+ GdkEventButton *event)
+ {
+ GET_PRIV(widget)->current_pointer = NULL;
+ ...
+ }
+
+ static gboolean
+ my_widget_motion_notify (Gtkwidget *widget,
+ GdkEventMotion *event)
+ {
+ if (gdk_event_get_device (event) !=
+ GET_PRIV(widget)->current_pointer)
+ return FALSE;
+
+ ...
+ }
+ </programlisting>
+ </example>
+ </section>
+
+ <section>
+ <title>Grabs</title>
+
+ <para>
+ Grabs are a mechanism to coerce a device into sending events to
+ a window, but with multidevice there's an other side of the coin,
+ how other devices are supposed to interact while the grab is in
+ effect.
+ </para>
+
+ <para>
+ The GdkGrabOwnership enum passed to gdk_device_grab() may be used
+ to block other devices' interaction. %GDK_OWNERSHIP_NONE applies
+ no restrictions, allowing other devices to interact, even with
+ the grab window. %GDK_OWNERSHIP_WINDOW blocks other devices from
+ interacting with the grab window, but they'll still be able to
+ interact with the rest of the application, whereas
+ %GDK_OWNERSHIP_APPLICATION will render the whole application
+ insensitive to input from other devices. Different devices may
+ have simultaneous grabs on the same or different windows.
+ </para>
+
+ <example>
+ <title>Grabbing as a result of an input event</title>
+ <programlisting>
+ gboolean
+ my_widget_button_press (GtkWidget *widget,
+ GdkEventButton *event)
+ {
+ GdkDevice *pointer, *keyboard;
+
+ pointer = gdk_event_get_device ((GdkEvent *) event);
+ keyboard = gdk_device_get_associated_device (pointer);
+
+ /* Grab both keyboard/pointer, other devices will be
+ * unable to interact with the widget window meanwhile
+ */
+ gdk_device_grab (pointer,
+ gtk_widget_get_window (widget),
+ GDK_OWNERSHIP_WINDOW,
+ ...);
+ gdk_device_grab (keyboard,
+ gtk_widget_get_window (widget),
+ GDK_OWNERSHIP_WINDOW,
+ ...);
+
+ return FALSE;
+ }
+ </programlisting>
+ </example>
+
+ <para>
+ For GTK+ grabs, there's only a boolean value, equivalent to
+ %GDK_OWNERSHIP_NONE and %GDK_OWNERSHIP_WINDOW, but the mechanism
+ is quite similar.
+ </para>
+
+ <para>
+ Once the device is grabbed, there may be different situations
+ that could break the grabs, so the widget needs to listen to
+ #GdkGrabBrokenEvent and the #GtkWidget::grab-notify signal to
+ handle these situations.
+ </para>
+
+ <example>
+ <title>Handling broken grabs</title>
+ <programlisting>
+ static gboolean
+ my_widget_grab_broken (GtkWidget *widget,
+ GdkEventGrabBroken *event)
+ {
+ MyWidgetPrivate *priv = GET_PRIV (widget);
+
+ if (gdk_event_get_device (event) == priv->grab_pointer)
+ {
+ /* Undo state */
+ ...
+ priv->grab_pointer = NULL;
+ return TRUE;
+ }
+
+ return FALSE;
+ }
+
+ static void
+ my_widget_grab_notify (GtkWidget *widget,
+ gboolean was_grabbed)
+ {
+ MyWidgetPrivate *priv = GET_PRIV (widget);
+
+ if (gtk_widget_device_is_shadowed (widget, priv->grab_device))
+ {
+ /* Device was "shadowed" by another widget's grab,
+ * release and undo state
+ */
+ ...
+ priv->grab_pointer = NULL;
+ }
+ }
+ </programlisting>
+ </example>
+ </section>
+
+ <section>
+ <title>Handling multipointer</title>
+
+ <para>
+ Widgets do react by default to every virtual device, although
+ by default they are set in a compatibility mode that makes them
+ behave better with multiple pointers, without necessarily
+ being multipointer aware.
+ </para>
+
+ <para>
+ This compatibility mode most notably disables per-device
+ enter/leave events, so these are stacked, and the crossing
+ events are only emitted when the first pointer enters the
+ window, and after the last pointer leaves it. This behavior
+ is controlled through gtk_widget_set_support_multidevice()
+ </para>
+ </section>
+
+ <section>
+ <title>Reading device axis values</title>
+
+ <para>
+ Button and motion events provide further information about
+ the device axes' current state. Note the device axes are
+ hardware and driver dependent, therefore the set of axes
+ is not set in stone, although there are a few more common ones.
+ </para>
+
+ <example>
+ <title>Getting to know the axes provided by a device</title>
+ <programlisting>
+ carlos sacarino:~$ xinput list "Wacom ISDv4 E6 Pen stylus" |grep "Label"
+ Label: Abs X
+ Label: Abs Y
+ Label: Abs Pressure
+ Label: Abs Tilt X
+ Label: Abs Tilt Y
+ Label: Abs Wheel
+ </programlisting>
+ </example>
+
+ <example>
+ <title>Getting an axis value</title>
+ <programlisting>
+ gboolean
+ my_widget_motion_notify (GtkWidget *widget,
+ GdkEventMotion *event)
+ {
+ GdkAtom *label_atom;
+ gdouble pressure;
+
+ label_atom = gdk_atom_intern_static_string ("Abs Pressure");
+ gdk_device_get_axis_value (gdk_event_get_device ((GdkEvent *) event),
+ event->axes, label_atom, &pressure);
+
+ /* Do something with pressure */
+ ...
+
+ return TRUE;
+ }
+ </programlisting>
+ </example>
+
+ <para>
+ All pointer devices report axes information, master and slave. to
+ achieve this, master pointers modify their list of axes at runtime
+ to reflect those of the currently routed slave, emitting
+ #GdkDevice::changed as the routed slave device changes.
+ </para>
+ </section>
+
+ <section>
+ <title>Dealing with slave (or floating) devices</title>
+
+ <para>
+ By default, GTK+ listens to all master devices, and typically
+ all slave devices will be attached to a master device. so
+ gdk_event_get_source_device() is the recommended way to deal
+ with the physical device triggering the event.
+ </para>
+
+ <para>
+ In more specialized setups, some devices could be floating
+ (i.e. tablets that don't route events through any virtual
+ pointer, but are expected to interact with drawing applications).
+ In that case, such specialized applications could want to interact
+ directly with the device. To do so, the device must be enabled,
+ and the widget wanting its events needs to add the event mask.
+ </para>
+
+ <example>
+ <title>Enabling events for a slave device</title>
+ <programlisting>
+ GdkDevice *device;
+
+ /* Gets the first device found with the given GdkInputSource */
+ device = get_device (gtk_widget_get_display (widget),
+ GDK_SOURCE_PEN);
+ gdk_device_set_mode (device, GDK_MODE_SCREEN);
+ gtk_widget_add_device_events (widget, device,
+ GDK_BUTTON_PRESS_MASK |
+ GDK_BUTTON_RELEASE_MASK |
+ GDK_POINTER_MOTION_MASK);
+ </programlisting>
+ </example>
+
+ <para>
+ After these calls, the widget would specifically receive events
+ from the physical device, regardless of it being floating or
+ connected to a master device. In this second case, and if you
+ want exclusive control of the device, you can temporarily detach
+ the stylus device from its master by doing a GDK grab on it.
+ </para>
+
+ <para>
+ For events coming directly from slave devices, both
+ gdk_event_get_device() and gdk_event_get_source_device() will
+ return the same device of type %GDK_DEVICE_TYPE_SLAVE or
+ %GDK_DEVICE_TYPE_FLOATING.
+ </para>
+
+ <note>
+ This is less useful than it used to be in GTK+2/XInput1, at least
+ for attached slaves, as there is gdk_event_get_source_device(),
+ and master devices' events provide axes information.
+ </note>
+ </section>
+ </section>
+
+ <section id="gtk-device-patterns-multitouch-widgets">
+ <title>Multitouch in GTK+ widgets</title>
+
+ <para>
+ Since version 3.4, GTK+ offers support for multitouch devices through a new
+ set of events and higher level tools like #GdkTouchCluster and
+ #GtkGesturesInterpreter.
+ </para>
+
+ <para>
+ If the widget does not have %GDK_TOUCH_MASK set in the event mask, it will
+ only be allowed to interact with the touch emulating pointer events, and will
+ only receive pointer events.
+ </para>
+
+ <para>
+ If the widget does have %GDK_TOUCH_MASK enabled, it will be able to receive
+ events of type %GDK_TOUCH_PRESS, %GDK_TOUCH_RELEASE and %GDK_TOUCH_MOTION,
+ which will be respectively emitted through the #GtkWidget::button-press-event,
+ #GtkWidget::button-release-event and #GtkWidget::motion-notify-event signals.
+ There may be multiple, simultaneous sequences of events, those will be
+ recognized and referenced by their touch ID. See gdk_event_get_touch_id().
+ </para>
+
+ <para>
+ #GtkWidget<!-- -->s may create GdkTouchCluster<!-- -->s via
+ gdk_window_create_touch_cluster(), those may be used to group
+ touch events together, which are notified through #GdkEventMultitouch,
+ this event will be emitted in #GtkWidget<!-- -->s through the
+ #GtkWidget::multitouch-event signal.
+ </para>
+
+ <para>
+ Widgets may also handle gestures being performed on them,
+ gtk_widget_enable_gesture() and gtk_widget_disable_gesture() are
+ provided as a simple API, although widgets may also create a
+ #GtkGesturesInterpreter and feed it events directly.
+ </para>
+ </section>
+
+ <section id="gtk-device-patterns-multitouch">
+ <title>Multitouch across a widget hierarchy</title>
+
+ <para>
+ Fully touch driven applications might not want to confine multitouch
+ operations within a single widget, but rather offer simultaneous
+ interaction with multiple widgets.
+ </para>
+ <para>
+ GTK+ is able to provide such experience, although it does not enable
+ %GDK_TOUCH_MASK by default on its stock widgets. If a widget meets the
+ following requirements, it is ready to be used in a multitouch UI:
+ </para>
+
+ <orderedlist>
+ <listitem>
+ The widget handles #GtkWidget::button-press-event, #GtkWidget::button-release-event
+ and #GtkWidget::motion-notify-event, and does something meaningful while the button 1
+ is pressed. If any explicit check on the event type being %GDK_BUTTON_PRESS,
+ %GDK_BUTTON_RELEASE or %GDK_MOTION_NOTIFY is performed, the event types
+ %GDK_TOUCH_PRESS, %GDK_TOUCH_RELEASE or %GDK_TOUCH_MOTION also need to be handled.
+ </listitem>
+ <listitem>
+ The widget relies on the implicit grab as long as the button press/touch is active,
+ GDK or GTK+ grabs would break the implicit grabs other touch sequences may have on
+ other widgets.
+ </listitem>
+ <listitem>
+ The widget does not require (or opts out) keyboard interaction while a touch is
+ active on it. Touch interaction does not necessarily bring the keyboard focus with it.
+ </listitem>
+ <listitem>
+ If the widget is only meant to interact with one touch sequence at a time (i.e.
+ buttons), it has to be able to discern and reject operations on any later touch
+ sequence as long as the touch it is interacting with remains active.
+ </listitem>
+ </orderedlist>
+
+ <para>
+ If a widget meets those requirements, enabling %GDK_TOUCH_MASK on it will suffice
+ to make it handle multitouch events in a way that doesn't disrupt other touch
+ operations.
+ </para>
+
+ <example>
+ <title>Enabling touch events on a widget</title>
+ <programlisting>
+ gtk_widget_add_events (widget, GDK_TOUCH_MASK);
+ </programlisting>
+ </example>
+
+ <note>
+ Not all GTK+ stock widgets are immediately suitable for handling touch
+ events, there could be even design reasons on some of those which render
+ them unsuitable.
+ </note>
+ </section>
+
+ <section id="gtk-device-patterns-recommendations">
+ <title>Recommendations</title>
+
+ <orderedlist>
+ <listitem>
+ Device operations often come up as a result of input events, favor
+ gdk_event_get_device() and gtk_get_current_event_device() before
+ gdk_device_manager_get_client_pointer().
+ </listitem>
+
+ <listitem>
+ Store the devices the widget is currently interacting with, handle
+ GdkEventGrabBroken and #GtkWidget::grab-notify to undo/nullify these.
+ </listitem>
+ </orderedlist>
+ </section>
+</chapter>
diff --git a/docs/reference/gtk/gtk-docs.sgml b/docs/reference/gtk/gtk-docs.sgml
index be7d3ce..bb983df 100644
--- a/docs/reference/gtk/gtk-docs.sgml
+++ b/docs/reference/gtk/gtk-docs.sgml
@@ -68,8 +68,9 @@
<xi:include href="xml/gtkstyle.xml" />
</part>
- <part id="multitouch">
- <title>Multitouch</title>
+ <part id="multitouch-and-multidevice">
+ <title>Interacting with input devices</title>
+ <xi:include href="xml/device-interaction-patterns.xml" />
<xi:include href="xml/gtkgesturesinterpreter.xml" />
</part>
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]
