Metacity Proposal: Grouping Windows

[Ryan McDougall <ryan mcdougall telusplanet net>]

This is a slightly more proof-read version of what I have already sent
to desktop-devel

Proposal to Add Grouping to Metacity, and some Rationale

0. Abstract

Solution presented for solving problems associated with window
management by allowing users to categorically organize windows by
defining custom window groups, and performing window transformations on
the groups as if they were true objects in "desktop space". The purpose
of this proposal is to improve the spatial properties of Metacity so
that it fits into a long term goal of spatial consistency within the
GNOME UI.

1. Problem specification and background

The Windows-Icons-Menus-Pointer paradigm has given our computers
excellent spatial multitasking abilities. Unfortunately like the clutter
of paperwork that occasionally builds up on our real world desks, a
profusion of windows can mean a real nuisance or even hindrance to
usability due to manual window management. 

1.1. The problem in Linux/GNOME platform

1.1.1 Window Management with Virtual Desktops

The traditional Unix approach to window organization is multiple
desktops. This technique helps us to categorize our windows into
meaningful groups, and reduces many window management tasks into
clicking on a desktop pager. However the setup and management of those
desktops is entirely manual, and does not allow one to interact with the
desktop as if it was a true, first class object in “desktop space” -- we
can't pull them or push them, resize them, mould them, or make them hide
away.


Also, currently virtual desktops are very handy, but provide an
inconsistent spatial representation. With a real desk the entire extent
is always visible; it is a whole that is easily discover-able, and
consistent with all the other desks one has seen. With a virtual desk
one has to realize that clicking on the pager jumps you one of these
desktops, and all windows and actions on windows are mutually exclusive
of windows on other desktops. From a spatial point of view, jumping to a
new desktop might as well be jumping to a new dimension, therefore its
inconsistent with physical reality. 


More abstractly, the concept of multiple "virtual" desktops may be
confusing to new users:

A random, non-scientific sample of 20 junior Computer Science students
at a major university running Ximian GNOME 1.4 on Sun workstations,
shows that only 3 users made use of more than one desktop. All were
running multiple tasks/windows (usually emacs, terminal, mozilla). When
5 students were asked why they weren't using the extra desktops, three
didn't know what that they existed, and two didn't consider it worth the
effort to maintain multiple desktops. This is purely anecdotal and
*unscientific*, but hopefully illustrative.


With the people I looked at above, the issue seems to be about the
discover-ability of virtual desktops. It doesn't seem intuitive *enough*
for them to easily grasp it even though they see the pager. Additionally
they will even see their peers using multiple desktops, yet they appear
to have no desire to make use of the same feature. 


Virtual Desktops are basically passive bins which hold windows and thus
can categorize, but they can be manipulated as spatial objects; and I
while I have no intention of removing them from GNOME, I think we can do
better. What that I think my survey shows is that people *like* to
organize their windows into categories that are meaningful to them. What
I propose isn't to remove the ability to organize, but take that ability
to the next level by shifting the paradigm slightly. 

1.1.2 Document-Tabbed UIs

Recent UI changes in GNOME, have lead to a resurgence in interest for a
solution to manage the proliferation of windows that can happen in any
spatial environment.

The popularity of Tabbed and MDI UIs in some applications is a direct
result of a user's annoyance with the current status of window
management. Tabbing allows the user to not only automatically group
document windows according to application, and thus according to user
meaningful types (the GNOME task bar currently has such a grouping
feature for multiple similar windows), but allows the user to apply
windowing transformations to all document windows at once. The
interaction is not passive, as a virtual desktop, rather one can
actively transform all windows as a whole group. Currently Tabbed
applications allow one to transform a set of document windows by issuing
commands to the parent, ie "resize GEdit (and by implication all sub-
documents currently open therein)", and has become a crowd favourite. 

The reasons are simple, organization is automatic: 

1. Each sub-document is owned by the parent app, not free-floating or
manually attached to the parent. The parent app represents a semantic
whole, of which the sub-documents are an integral part.

2. The application provides an easy to understand, spatially consistent
UI metaphor for accessing the sub-documents.

3. Any window management action taken on the parent applies uniformly to
the sub-document windows.


1.1.3 Spatial Nautilus

Also, the spatial behaviour of the new Nautilus has highlighted the
existing problems with manually managing UI objects. Each spatial window
of Nautilus adds to the complication of managing that many windows, and
users are rightly concerned. However if we revert or break spatial
Nautilus, the symptoms will be lessened, but the problem will never go
away.

1.2. Existing Solutions on other platforms

Apple has taken a very innovative approach, called expose, with the
technology in OSX which allows the user to apply a couple predefined
transformations to the open windows on a desktop. One thumbnails all
open windows, the second packs all open windows into the screen, the
third packs all open windows belonging to the foreground application
into the screen. While these methods can only categorize windows by
foreground application (3rd method), unlike virtual desktops which can
be arbitrarily categorized, the significant improvement with expose is
that with one keyboard sequence or mouse gesture, one can act upon the
windows in an easily understandable, well-define, spatial way.

1.3. Current Status on Linux/GNOME platform

While XFree lacks the technical ability to graphically mimic Apple's
expose feat, there is nothing stopping the WM from attempting a similar
feature set, although perhaps without the prettiness.

2. Proposal and Implementation

2.1. Solution to Managing Windows

My proposal is two-step, and as far as I can tell, presents no major
technical hurdles: 

First, users need a way to more intuitively group their windows, and
groups of windows, into user-meaningful categories. Human understanding
heavily uses hierarchical categorization, and virtual desktops and task
bars show that we like to organize our windows into groups that hold
meaning to us as individuals. Allowing users to "grab" a set of windows
or set of groups of windows, and form them into a semantic group will
help us do so in a spatially consistent way. The method of grabbing a
set of windows should be more intuitive (than Virtual Desktops) for any
user who has performed a similar operation in selecting a bunch of files
via and modern file manager.

Second, users need a way to specify transformations on those groups of
windows. Expose provides such transformations, but they are limited to
three kinds, and have no concept of application groupings, nor any
understanding of the irregular grouping that make sense to us. Using the
arbitrary conceptual groups defined above, and arbitrary transformations
on said groups, we can mimic in desktop space the behaviour our hands,
arms, or other tools might have in real space. Some useful
transformations would be to “Tile all selected windows”, “Iconify all
selected windows”, or “Move all selected window to the left”, etc.

2.2 Solution Implementation Details

The default group is "All Windows or Sub-groups", and can be acted upon
with keyboard shortcuts or mouse gestures when no Sub-groups are in
Focus (ala expose).

2.2.1 Managing Group Lifetimes

The Metacity menu will provide three grouping options, tentatively
called:

1. "Ungroup"

This option changes the mouse icon to a cut lasso (or similar). If the
user then clicks on any grouped window, the component sub-windows or
sub-groups become disassociated and no longer behave as a group.

2. "Group Together"

This option changes the mouse icon to a lasso (or similar), with a user
defined bounding box underneath. Any window or group of windows that
intersects with the bounding box becomes part of the new group. This
operation is similar to “shift-selecting” a group of icons in a file
manager.

Metacity then forms a new parent window surrounding the union of all
windows and sub-groups as they preexisted in desktop space. Any
windowing transformation done on the new parent window applies to all
sub-windows and sub-groups. A resize on the parent causes the sub-
windows and sub-groups to resize in proportion.

3. "Stack Together"

This option changes the mouse icon to a lasso (or similar), with a user
defined bounding box underneath. Any window or group of windows that
intersects with the bounding box becomes part of the new group. This
operation is similar to “shift-selecting” a group of icons in a file
manager.

Metacity then forms a new parent window, with a horizontal widget-space
below it to hold tabs that represent all the sub-windows and sub-groups
in the new group. Clicking on a tab unmaps the first open tab, and maps
the selected tab.

Also, we want to allow applications to automatically add new documents
to the Metacity Tab bar. This is probably the most technically
challenging since it would requite a change to the WM API, as well as
specs and or protocols. Factoring out the Tabbing behaviour to the
window manager we can reduce application complexity, and retain the
convenience of tabbed organization for applications that benefit from
it.

Tabbed Grouping is only recommended for applications that open a lot of
similar documents, such as a web browser, text browser, or spatial
nautilus. Tabs are conceptually consistent on documents and little else,
but in the case of spatial Nautilus, tabbed elements would look very
much like the path buttons on Seth Nickell's File Chooser UI.

2.2.2 Transforming Groups

1. "Group Together" Mode

Any translation of the parent window by (x,y) causes the sub-windows and
sub-groups to be recursively translated by (x,y). Any translation of one
of the sub-windows is independent of the parent or other sub-windows. If
a translated sub-window shrinks or grows the original bounding box, then
the parent window shrinks or grows to contain the new position of the
sub-window.

Any minimize/unminimize/maximize of the parent window causes the sub-
windows and sub-groups to be recursively minimized/unminimized/
maximized. Any minimize/unminimize/maximize of one of the sub-windows
causes the window to "roll up"/"unroll" respectively, since disappearing
or exploding windows would break the conceptual unity of the group.

Any resize of the parent window by (x%, y%) causes the sub-windows and
sub-groups to be recursively scaled proportional to the parent. This
part will be tricky, keeping the scaling right, and I don't have an
algorithm in mind just yet.

2. "Stack Together" Mode

Any translation of the parent window by (x,y) causes the sub-windows and
sub-groups to be recursively translated by (x,y).

Any minimize/unminimize/maximize of the parent window causes the sub-
windows and sub-groups to be recursively minimized/unminimized/
maximized.

Any resize of the parent window resizes only the currently active tab.
Window sizes are independent of the parent.

3. Interactions and Use Cases

3.1 Spatial Nautilus in Tabbed mode

The user is searching for a file: User opens the first Nautilus window
en route to the desired file. In the process of "drilling down" into a
directory structure various new spatial windows open, and register as
the current tab in the current parent window. The previous windows exist
and are kept in the Nautilus Group's tabs, but are unmapped so only the
current working folder is visible. Note: this is equivalent to the
manual "Stack" grouping mode.

Benefits: No explosion of windows. All folders exist, and are spatially
"there", but are grouped in one transformable entity. They are grouped
according to the path taken through the file system, which is a
meaningful categorization, and thus there is a history of all the
folders visited representing a continuous train of exploration. In the
real world Folders are not Documents, although real folders are often
tabbed and arranged in filing cabinets.

Drawbacks: Fill in here. ;)

3.2 Coding Session with Emacs, gnome-terminal, Mozilla, and Evolution

The user needs to group 4 distinct applications: After opening all four
apps, the user arranges them about the desktop in the way that is most
comfortable. The user then clicks on a Metacity window's "Group
Together" command, and draws the bounding box around the four apps to
group them. The user can now resize, or move the 4 as if they were one
window.

Benefits: Although the four apps are unrelated by development, a
programmer will logically think of this set as one unit of windows
needed to perform his task. The window manager bridges this gap by
allowing the user to specify his unique applications sets.

Drawbacks: Manual creation of the window group.

3.3 Transformations for Managing Ungrouped Windows

The user needs to find a certain PDF document-window among many open
windows: The default group is the global group of "All windows" so the
user causes, via a certain keyboard shortcut, all windows to tile into
the root window. The user then can select the right PDF window by
reading the unobscured Title bar, and highlighting the choice with the
Tab button. (Behaviour is much like expose is now)

Benefits: User doesn't have to minimize all open windows, nor route
through a crowded task list for the correct title, but can use his
spatial intuition to "spread out" his open windows across his desk, and
choose the correct one from an overview perspective.

Drawbacks: XFree currently isn't able to quickly provide visual feedback
about what is in the window, like OSX can.

4. Whats left to discuss?


4.1 What some people have asked already.

“What makes you think users are more likely to use window grouping than
they are to use virtual desktops?”

I think they are more likely to use grouping for a number of reasons:

1. Its at least as useful as Virtual Desktops. With on average less
mouse/key
strokes one can group a whole set of windows and drag them to a new
workspace.

2. Its more useful than Virtual Desktops. Once grouped you can now
manipulate
the group as if it were a real window. In Stacked Grouping you can
manipulate a whole series of similar windows, such as spatial nautilus
windows, move, resize, minimize, etc. in one stroke. 

3. Its more spatially intuitive. The windows created by grouping are
first class objects which can be handled as such. Virtual Desktops are
just
passive bins which hold things. 



Cheers,
Ryan