[PATCH 2/2] Add extended frame synchronization protocol

From: "Owen W. Taylor" <otaylor fishsoup net>

To support an application marking the beginning and end of the
frame, add support for an extended frame synchronization protocol
that extends the protocol of _NET_WM_SYNC_REQUEST to allow an
application to provide a counter that it updates to an odd
value to begin a frame and to an even value to end the frame.

On top of that are built:

 * A message _NET_WM_FRAME_DRAWN that is sent when the compositor
   has drawn the frame.

 * A message _NET_WM_FRAME_TIMINGS that is sent including extra
   frame timing information.

 * A property _NET_WM_SYNC_FENCES that can be combined with the
   counter value to find an appropriate fence to wait on.
 wm-spec/wm-spec.xml | 500 ++++++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 465 insertions(+), 35 deletions(-)

diff --git a/wm-spec/wm-spec.xml b/wm-spec/wm-spec.xml
index 8d83a94..debaea4 100644
--- a/wm-spec/wm-spec.xml
+++ b/wm-spec/wm-spec.xml
@@ -1773,28 +1773,327 @@ window when all updates are done. The application should not be generating many
 frames of content when only one of them is drawn to the output device.
-Limited synchronization of window manager and client drawing during resizing
-is possible without a compositing manager, and is briefly described below,
-but the remaining protocols only make sense when there is a compositing
-manager. These protocols are additionally designed with the assumption that
-the window manager and the compositing manager are the same process, and are
-not applicable to the case of a stand-alone compositing manager.
+Limited synchronization of window manager and client drawing during
+resizing is possible without a compositing manager, and described
+below under <link linkend="basic_synchronization">Basic
+Synchronization</link>, but the majority of protocols in this section
+only make sense when there is a compositing manager. These protocols
+are additionally designed with the assumption that the window manager
+and the compositing manager are the same process, and are not
+applicable to the case of a stand-alone compositing manager.
-  <sect2 id="NET_WM_SYNC_REQUEST">
-    <title>_NET_WM_SYNC_REQUEST</title>
+  <para>
+Both forms of synchronization identify "frames" of drawing using
+XSync extension <citation><link linkend="XSync">XSync</link></citation>
+counters. The XSync extension allows creating Counter objects that
+hold a 64-bit value. Applications can select to get events when
+the counter object changes or reaches a particular value.
+  </para>
+  <para>
+In the synchronization protocols, drawing is timed with reference to
+the output device's <firstterm>refresh cycle</firstterm>. An output
+device, such as a monitor, will typically read data from the frame
+buffer sequentially starting at the top of the displayed area, pause
+for a period known as the <firstterm>vertical blanking
+period</firstterm> (or vertical blanking interval) and then
+repeat. The vertical blanking period provides an opportunity to
+atomically change the screen contents without the risk of tearing, and
+is when a compositor using double-buffered drawing will swap
+buffers. The entire length of the process is called the
+<firstterm>refresh interval</firstterm>. For example, a monitor
+updating at 60 frames per second has a refresh interval of 1/60th of a
+  </para>
+  <para>
+A compositor may be managing multiple output devices with different
+refresh cycles, and windows may overlap multiple devices. This means
+when an application draws a frame, the point in time where that frame
+of drawing actually is displayed to the user may be different on
+different output devices. For each application frame, the compositor
+MUST identify a single output device that the frame is
+<emphasis>primarily</emphasis> displayed upon, and report timing
+information for the display of the frame with respect to that
+device. For frame updates that don't include the entire window, the
+chosen device MAY depend on the particular updated area, and the
+chosen device MAY change from frame to frame for this or other
+  </para>
+  <sect2 id="basic_synchronization">
+    <title>Basic Synchronization</title>
+    <para>
+The goal of basic synchronization is limited to coordinating redraws
+during interactive resizing.  A client indicates that it is willing to
+participate in basic synchronization by listing _NET_WM_SYNC_REQUEST
+in the WM_PROTOCOLS property of the client window and storing the XID
+of a XSync counter in the property _NET_WM_SYNC_REQUEST_COUNTER.
+This counter is known as the <firstterm>basic frame counter</firstterm>.
+    </para>
+    <para>
+Before resizing a window, the window manager sends a
+_NET_WM_SYNC_REQUEST message to the client window containing a value
+that the application stores in the basic frame counter when it is done
+handling the ConfigureNotify event resulting from the resize. This
+allows the window manager to know that it can move on to the next step
+of resizing without getting ahead of the client. If the window manager
+is a compositing manager it may also choose to freeze redrawing of the
+window until it sees the change in counter value, so that the resized
+window frame and window are drawn as a single unit.
+    </para>
+  </sect2>
+  <sect2 id="extended_synchronization">
+    <title>Extended Synchronization</title>
+    <para>
+Extended synchronization provides a more general framework for redraw
+coordination, including spontaneous application updates as well as
+resizing. A client indicates its willingness to participate in the
+extended form of frame synchronization by listing _NET_WM_SYNC_REQUEST
+in WM_PROTOCOLS and including <emphasis>two</emphasis> XSync counters
+in the property _NET_WM_SYNC_REQUEST_COUNTER, the basic frame counter
+and an <firstterm>extended frame counter</firstterm>. A window manager
+indicates that it will participate by listing _NET_WM_FRAME_DRAWN in
+    </para>
+    <para>
+In extended synchronization, the basic frame counter is unused.  The
+extended frame counter is updated in response to _NET_WM_SYNC_REQUEST
+messages, but can can additionally be updated spontaneously by the
+client to indicate the beginning or end of a frame of drawing. The
+beginning of a frame of drawing is indicated by an increment to an odd
+number, and the end of a frame of drawing is indicated by an increment
+to an even value.  For each frame marked this way, a client will
+receive _NET_WM_FRAME_DRAWN and _NET_WM_FRAME_TIMINGS messages from
+the compositor.
+    </para>
+    <para>
+Rationale: using the same counter for basic and extended
+synchronization would cause problems when switching from a window
+manager that supports only the basic synchronization to one supporting
+extended synchronization - since in one case the window manager
+determines the counter values and in the other case the client
+determines the counter values, there would be race conditions during
+the switch. Packing both counters into a single property increases
+efficiency when initially managing a window.
+    </para>
+  </sect2>
+  <sect2 id="frame_timing_algorithm">
+    <title>Compositor frame timing algorithm</title>
+    <para>
+When an application is displaying changing content, such as animated
+transitions, videos, or user interface elements that respond to the
+user's mouse input, there are three primary measures of quality:
+<firstterm>Frame rate</firstterm> - the number of frames that are
+drawn per second. <firstterm>Latency</firstterm> - the time between
+when a frame is generated and when it is displayed to the user. Moderate
+latency will cause dragged objects to lag behind the cursor, higher
+latency will cause apparent discrepencies to the user between when an
+action is taken and when it is displayed. In some cases, such as the
+synchronization of audio and video, a known latency can be compensated
+for. <firstterm>Jitter</firstterm> - the difference in latency across
+frames. Jitter causes the appearance of stuttering, uneven motion.
+    </para>
+    <para>
+The choice of algorithm for scheduling redraws in the compositor will
+involve tradeoffs between these quality measures. In particular
+there is a tradeoff between latency and jitter. If a compositor draws
+as soon as it receives a new frame from any application, this will
+result in minimum latency with large amounts of jitter - a frame may
+be processed immediately and drawn with minimal delay, but if
+the compositor is busy processing a redraw triggered
+by another client, there will be additional latency.  In order to
+present consistent latency to applications, an alternate approach is
+recommended: after receiving a new frame, the compositor waits for a
+fixed point in the refresh cycle before starting the redraw. By making
+sure that each frame of drawing is finished with some time to spare before this
+point, an application achieves a latency that is both known and
+    </para>
+    <para>
+The approach of waiting to a fixed point in the refresh cycle can
+backfire if an application is unable to achieve its target frame rate:
+the delay before the compositor draws will waste time that could have
+been spent drawing and cause unnecessary synchronization to a
+sub-multiple of the system's refresh rate. For this reason, in the
+protocols described below, frames can be marked as
+<firstterm>urgent</firstterm>.  In response to an urgent frame, a
+compositor SHOULD draw as quickly as possible and send a
+_NET_WM_FRAME_DRAWN message so the application can draw the next
+frame. One approach to deciding which frames to mark urgent is for the
+application to note whether it sleeps after receiving
+_NET_WM_FRAME_DRAWN before starting the next frame. If not, the next
+frame is marked urgent.
+    </para>
+    <para>
+The following algorithm is <emphasis>recommended</emphasis> for
+compositors. It is however, not <emphasis>required</emphasis>.
+    </para>
+  <listitem>
+    <para>When the compositor receives a XDamageNotify event:</para>
+    <itemizedlist>
+      <listitem><para>If it is part of an urgent frame, schedule an immediate redraw</para></listitem>
+      <listitem><para>Otherwise, schedule a redraw for the next redraw point. Redraw points
+      occur <replaceable>frame_delay</replaceable> milliseconds after the start of the vertical blanking period.</para></listitem>
+    </itemizedlist>
+  </listitem>
+  <listitem><para>
+    If a redraw is scheduled for time T, and and the swap has not yet completed at time T, redraw immediately when the swap completes
+  </para></listitem>
+    <para>
+The choice of <replaceable>frame_delay</replaceable> is up to the
+compositor. The minimum event-handling latency that an application can
+achieve is <replaceable>application_frame_draw_time</replaceable> +
+<replaceable>refresh_interval</replaceable> -
+<replaceable>frame_delay</replaceable>. A larger value will reduce
+latency slightly, but increases the chance that the compositor won't
+be done drawing the frame by the vertical blanking period, and
+it will be displayed a refresh cycle late. Using
+0 for <replaceable>frame_delay</replaceable> for delay is not
+recommended, since that will produce pessimized latency for legacy GL
+clients that swap buffers during the vertical blanking period.
+Instead a small value such as 2 milliseconds is recommended.
+    </para>
+  </sect2>
+  <sect2>
+    <title>High precision timestamps</title>
+    <para>
+When exchanging timing information about drawing, it is useful for
+clients to be able to have a higher-precision than the millisecond
+precision of X server timestamps. To allow for greater precision,
+intervals and timestamps for protocols in this section are represented
+in microseconds. For timestamps, the timestamp has the form
+(<replaceable>x_server_timestamp</replaceable>) * 1000 + <replaceable>microsecond_value</replaceable>.
+    </para>
+    <para>
+The window manager MAY produce these <firstterm>high precision
+timestamps</firstterm> by periodically determining an offset between
+the server time and a local system time. The server time can be
+determined by the standard method of changing a dummy property and
+observing the timestamp in the PropertyNotify event.  For the system
+time it is preferred to use a monotonic time, such as the time given
+by clock_gettime(CLOCK_MONOTONIC, ...), rather than a real time, such
+as that given by gettimeofday(). If this method of tracking an offset
+is used, the accuracy of passing a time between two clients will be
+limited by the precision of the X server time, and the round-trip time
+for obtaining the X server time.
+    </para>
+    <para>
+Alternatively, a client MAY observe that X server times correspond to
+a specific method of obtaining the local time, such as using
+gettimeofday() or clock_gettime(). In this case, high precision times
+can be generated directly, and two clients such will be able to
+exchange timestamps with sub-millisecond accuracy. (This technique is
+only applicable to clients running on the same machine as the X
+    </para>
+  </sect2>
+  <sect2>
+    <title>_NET_WM_SYNC_REQUEST_COUNTER</title>
+    <programlisting><![CDATA[
+    <para>
+The _NET_WM_SYNC_REQUEST_COUNTER property is set by the application on a toplevel
+window. It contains either one or two XSync counter IDs, depending on whether
+the client supports only basic synchronization or both basic and extended
+    </para>
+    <para>
+If the client supports only basic synchronization, then it puts only the
+basic frame counter ID in the _NET_WM_SYNC_REQUEST_COUNTER property. The initial
+value of basic frame counter is not defined by the specification. The window
+manager MAY set the value of the basic frame counter at any time, and,
+if it chooses t use basic synchronization, MUST do so when it first manages
+a new window.
+    </para>
+    <para>
+If the client supports extended synchronization, then the client
+creates both a basic frame counter and an extended frame counter, and
+puts both IDs in _NET_WM_SYNC_REQUEST_COUNTER, with the first element
+of the property being the basic frame counter ID and the second
+element the extended frame counter ID. The client sets the extended
+frame counter to an initial value before before changing the window
+from the withdrawn state, and the client MAY update the value at any
+    </para>
+    <para>
+To indicate the beginning of a frame, the client increases the value
+of the extended frame counter to an odd value. At this point, the
+window is an a "frozen" state and the compositing manager SHOULD NOT
+redraw the window in response to XDamageNotify events. If the
+compositing manager needs to redraw for other reasons, such as a
+change to a different window that overlaps the frozen window, it MAY
+redraw the window. To indicate the end of the frame the client
+increases the value of the extended frame counter to an even value. At
+this point the window is no longer frozen, and the compositor SHOULD
+redraw the window and send _NET_WM_FRAME_DRAWN and
+_NET_WM_FRAME_TIMINGS messages. The window manager SHOULD redraw the
+window and send the messages even if no damage events were received.
+    </para>
+    <para>
+Rational: the reason for redrawing the window even if no damage events have
+been received is so that there is consistent timing for each frame,
+even if one frame happens to involve no changes to the window. It is recommended
+to behave as if a damage event was received containing only a single pixel.
+    </para>
+    <para>
+To distinguish urgent from non-urgent frames, two different patterns of
+increases are used. (See <xref linkend="frame_timing_algorithm"/> for
+the definition of urgent frames.) To mark an urgent frame the odd value
+is chosen to have a value such that <replaceable>v</replaceable> % 4 = 3, then
+the counter is increased by 1 to give the even value. To mark a
+non-urgent frame the odd value is chosen to have a value such that
+<replaceable>v</replaceable> % 4 = 1, then
+the counter is increased by 3 to give the even value.
+    </para>
+  </sect2>
+  <sect2>
+    <title>_NET_WM_SYNC_FENCES</title>
+    <para>
+On some systems with loose synchronization between different clients
+using the graphics system, drawing immediately after XDamageNotify
+events are received does not work properly, and a GL-based compositing
+manager must also insert GL-level synchronization to ensure correct
+drawing. Because there is no way of knowing whether that is the case
+on any particular system, _NET_WM_SYNC_FENCES SHOULD always be
+set by applications and compositors MUST synchronize drawing
+either using _NET_WM_SYNC_FENCES or the alternate method described
+    </para>
+    <para>
+The _NET_WM_SYNC_FENCES property is set on a toplevel by an
+application and contains a list of XSync fence objects. Before ending
+a frame by setting the XSync counter to an even value N, the
+application uses XSyncTriggerFence() to trigger the fence object
+stored in the property at index (N / 4) % L, where L is the number of
+counters in property. If an application waits for the
+_NET_WM_SYNC_DRAWN message before starting a new frame, 2 is a
+sufficiently large value for L.
+    </para>
-This protocol uses the XSync extension (see <ulink
-protocol specification</ulink> and <ulink
-the library documentation</ulink>) to let client and window manager
-synchronize the repaint of the window manager frame and the client
-window. A client indicates that it is willing to participate in the
-protocol by listing _NET_WM_SYNC_REQUEST in the WM_PROTOCOLS property
-of the client window and storing the XID of an XSync counter in the
-property _NET_WM_SYNC_REQUEST_COUNTER. The initial value of this
-counter is not defined by this specification.
+When the window is mapped, the GL-based compositor SHOULD import the
+fences in _NET_WM_SYNC_FENCES as GL sync objects using the
+ImportSyncEXT() procedure from <citation><link
+After receiving a frame update ending with counter value N, the next
+time that the window is redrawn, the compositing manager calls
+glWaitSync() on the fence at index (N / 4) % L.
+    <sect3>
+      <title>Alternate method</title>
+      <para>
+If an application window doesn't export _NET_WM_SYNC_FENCES, or draws
+outside a frame update, the compositing manager still MUST ensure
+correct synchronization. If the compositing manager has received
+XDamageNotify events that are not synchronized by _NET_WM_SYNC_FENCES, then
+before drawing the screen, the compositing manager MUST use
+XSyncTriggerFence() to trigger an X fence it has created itself, and then
+wait for that fence using glWaitSync().
+      </para>
+    </sect3>
+  </sect2>
+  <sect2 id="NET_WM_SYNC_REQUEST">
+    <title>_NET_WM_SYNC_REQUEST</title>
 A window manager uses this protocol by preceding a ConfigureNotify
 event sent to a client by a client message as follows:
@@ -1806,31 +2105,139 @@ message_type = WM_PROTOCOLS
 format = 32
 data.l[0] = _NET_WM_SYNC_REQUEST
 data.l[1] = timestamp
-data.l[2] = low 32 bits of the update request number
-data.l[3] = high 32 bits of the update request number
-other data.l[] elements = 0
+data.l[2] = low 32 bits of a frame counter value
+data.l[3] = high 32 bits of a frame counter value
+data.l[4] = 1 if the the extended frame counter should be updated, otherwise 0
-After receiving one or more such message/ConfigureNotify pairs, and
-having handled all repainting associated with the ConfigureNotify
-events, the client MUST set the _NET_WM_SYNC_REQUEST_COUNTER to the 64
-bit number indicated by the data.l[2] and data.l[3] fields of the last
-client message received.
+If data.l[4] is 0, then after receiving one or more such
+message/ConfigureNotify pairs, and having handled all repainting
+associated with the ConfigureNotify events, the client MUST set the
+basic frame counter to the frame counter value from the last client
+message received.
+    </para>
+    <para>
+The frame counter value in the client message is determined by the
+window manager subject to the restriction that it MUST NOT be 0. The
+number is generally intended to be incremented by one for each message
+    </para>
+    <para>
+If data.l[4] is 1, then after receiving one or more such
+message/ConfigureNotify pairs, and having handled all repainting
+associated with the ConfigureNotify events, the client MUST set the
+extended frame counter to a value <emphasis>greater than</emphasis>
+the frame counter value in the last client message received.  This will
+normally be done as part of indicating the end of a frame of drawing.
+The exact value used is chosen so that the fences listed in
+_NET_WM_SYNC_FENCES are used in rotation.
+    </para>
+    <para>
+The frame counter value in the client message is determined by the
+window manager based on the most recent value it has seen for the
+extended frame counter. The window manager SHOULD retrieve
+this value when managing the window, and monitor subsequent changes
+to the value by creating an XSync Alarm object. The update request
+number in the client message SHOULD be chosen as
+<replaceable>last_seen_value</replaceable> + 240.
+    </para>
+    <para>
+Rationale: if the client is continually redrawing, then the last
+seen value may be out of date when the window manager sends the message.
+Picking a number that is 240 later would allow for 1 second of frames
+at 60fps. In the normal case client will draw only one more frame before
+waiting for _NET_WM_FRAME_DRAWN.
 By using either the Alarm or the Await mechanisms of the XSync
 extension, the window manager can know when the client has finished
-handling the ConfigureNotify events. The window manager SHOULD not
+handling the ConfigureNotify events. The window manager SHOULD NOT
 resize the window faster than the client can keep up.
+  </sect2>
+  <sect2>
+    <title>_NET_WM_FRAME_DRAWN</title>
-The update request number in the client message is determined by the
-window manager subject to the restriction that it MUST NOT be 0. The
-number is generally intended to be incremented by one for each message
-sent. Since the initial value of the XSync counter is not defined by
-this specification, the window manager MAY set the value of the XSync
-counter at any time, and MUST do so when it first manages a new
+This client message allows a client to know when an update it has
+created has been drawn to the screen by the compositing window
+manager.  If the compositing manager lists this property in
+_NET_SUPPORTED, then the compositing manager MUST send a client that
+supports extended synchronization a _NET_WM_FRAME_DRAWN client message
+immediately after finishing the redrawing that results from an
+application frame. If an application draws several frames without
+waiting for _NET_WM_FRAME_DRAWN, the compositing manager MAY send only
+one _NET_WM_FRAME_DRAWN for the last frame. The contents of the
+client message are as follows:
+    </para>
+    <programlisting><![CDATA[
+type = ClientMessage
+window = the respective client window
+message_type = _NET_WM_FRAME_DRAWN
+format = 32
+data.l[0] = low 32 bits of the extended frame counter value
+data.l[1] = high 32 bits of the extended frame counter value
+data.l[2] = low 32 bits of high precision timestamp
+data.l[3] = high 32 bits of high precision timestamp
+data.l[4] = 0
+    <para>
+The timestamp indicates that time at which the compositing manager
+finished submitting drawing for entire scene to the graphics system.
+(If that time is not available, the compositing manager MAY approximate
+it by the time at which the message is sent.) The actual completion
+of drawing will occur at some time later than the indicated time.
+    </para>
+    <para>
+In addition to sending one message at the end of each frame, the
+compositing manager MUST send one _NET_WM_FRAME_DRAWN message for each
+newly mapped window that supports extended synchronization. If the
+initial value of the frame counter is odd, then the window starts in
+the frozen state, and the message is sent as per normal after the
+frame ends. If the initial frame value is even, the message is
+sent after the window is first drawn.
+    </para>
+  </sect2>
+  <sect2>
+    <title>_NET_WM_FRAME_TIMINGS</title>
+    <para>
+This message provides information about the timing of a previous frame;
+it is sent subsequent to the _NET_WM_FRAME_DRAWN message for the frame
+once the window manager has obtained all available timing information.
+    </para>
+    <programlisting><![CDATA[
+type = ClientMessage
+window = the respective client window
+message_type = _NET_WM_FRAME_TIMINGS
+format = 32
+data.l[0] = low 32 bits of the extended frame counter value
+data.l[1] = high 32 bits of the extended frame counter value
+data.l[2] = presentation time offset
+data.l[3] = refresh interval
+data.l[4] = frame_delay
+    <para>
+The presentation time offset is a 32-bit signed offset in microseconds
+from the timestamp in the _NET_WM_FRAME_DRAWN message. It may be 0 if
+no presentation time is available. The presentation time indicates the
+time of the end of the vertical blanking period after which the frame
+contents will be scanned out to the output device. (If this exact time
+is not available, the composite manager MAY provide a nearby time such
+as the time at which buffers are swapped.)
+    </para>
+    <para>
+The refresh interval a 32-bit unsigned number indicating the
+refresh interval in microseconds. It may be 0 if the refresh interval is
+not available.
+    </para>
+    <para>
+The frame delay is a 32-bit unsigned number indicating the point in the
+refresh interval where the compositor will start drawing if it previously
+received damage, as a number of microseconds after the end of the
+vertical blanking period.
+See <xref linkend="frame_timing_algorithm"/> for details.
+The flag value 0x80000000 indicates that the compositor uses a different
+frame timing algorithm and no meaningful value is available. All other
+values with the high-bit set are reserved.
@@ -2239,6 +2646,29 @@ int net_get_hostname (char *buf, size_t maxlen)
+    <varlistentry>
+    <term>[XSync]</term>
+    <listitem>
+    <para id="XSync">
+      Tim Glauert, Dave Carver, Jim Gettys, David P. Wiggins, and James Jones,
+      "X Synchronization Extension Protocol, X Consortium Standard" (Version 3.1)
+      <ulink url="http://www.x.org/releases/X11R7.7/doc/xextproto/sync.html";>
+      http://www.x.org/releases/X11R7.7/doc/xextproto/sync.html</ulink>
+    </para>
+    </listitem>
+    </varlistentry>
+    <varlistentry>
+    <term>[x11_sync_object]</term>
+    <listitem>
+    <para id="x11_sync_object">
+      Piers Daniell, Pierre-Loup Griffais, James Jones,  Aaron Plattner,
+      "EXT_x11_sync_object".
+      <ulink url="http://www.opengl.org/registry/specs/EXT/x11_sync_object.txt";>
+      http://www.opengl.org/registry/specs/EXT/x11_sync_object.txt
+      </ulink>
+    </para>
+    </listitem>
+    </varlistentry>

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]