Mousetweaks: integrate into GNOME Shell or keep in its current form?
- From: Magdalen Berns <magdalenberns gmail com>
- To: gnome-accessibility-devel gnome org, gnome-shell-list gnome org
- Subject: Mousetweaks: integrate into GNOME Shell or keep in its current form?
- Date: Thu, 21 Nov 2013 00:18:45 -0000
Hi all,
Sorry for the epic email. I was speaking to mclasen and halfline about
the mousetweaks discussion I started in a private email thread with the
MT developers Gerd and Francesco and, rather than ccing people at random
intervals it seems more appropriate to forward to the gnome-shell and
accessibility lists so others can comment!
Hope the thread below makes sense, but just to summarise the discussion
it is about whether or not to build Mousetweaks into GNOME Shell as
there were a couple of of considerations which need to be thrashed out
before a decision can be arrived at.
Unfortunately I had to paste the thread from different parts or bits
would have got missed so let us know if there's anything too confusing
to figure out from the text below and I can try to help.
Magdalen
Magdalen Berns <magdalenberns gmail com>
22 Oct
to Gerd, Francesco
Hi Gerd, Francesco,
How are you both doing? I wonder if you got my email about mousetweaks a
while back? It may have got lost in the lists.
Thanks,
Magdalen
Gerd Kohlberger
23 Oct
to marmuta, me, Francesco
Hi Magdalen,
yeah I didn't see your mail until now. Sorry. Francesco and I haven't
really talked about the future of mousetweaks since your gsoc mails.
(congratulations by the way)
I think porting mousetweaks in its current form to wayland is not going
to work. To implement hover-click we would need 2 things: a) get global
mouse-motion events and b) synthesize/fake mouse button events. A
regular application can do neither under wayland. I can think of 2 ways
to solve this, either move everything into gnome-shell as you suggest,
or keep mousetweaks as a standalone daemon and use at-spi.
at-spi has API for global mouse-events (AtspiEventListener + "mouse:*"
events) and for faking mouse events (atspi_generate_mouse_event()), but
both operations don't work under wayland at the moment. Until those
things are implemented we can't start our port.
Reimplementing mousetweaks in gnome-shell would have a lot of
advantages. It would be easy to get mouse-events and the click-selection
window could be integrated directly into the chrome. It's currently
impossible to drag things around in the Activites overview, because you
can't change the hover-click type from within the overview. Another
advantage would be that we save a lot of code. Probably 90% of
mousetweaks only deals with being a daemon and getting various events,
the actual hover-click logic is only a tiny part.
There is however one problem, if all the features move into gnome-shell
then mousetweaks becomes a GNOME-only solution. At the moment
mousetweaks works fine on Unity, lxde and even KDE. It can also be used
as a service. For example the default on-screen keyboard on Ubuntu
(onboard) integrates mousetweaks into its interface. You can activate
hover-click and select the different click-types directly with the
keyboard.
I would actually prefer to just start working on a hoverclick.js
component for gnome-shell (or you could Magdalen if you like), but
loosing the onboard integration is a big problem. Francesco, what do you
think? I'm guessing you're not even running gnome-shell.
Cheers.
CC'ing marmuta (main onboard developer)
marmuta, what do you think about droping mousetweaks and also
implementing hover-click? The at-spi dependency is already there and it
might even be easier to do it ourselves than to do ipc with mousetweaks.
It would at least resolve the issues with hiding the click-selection
window.
Magdalen Berns <magdalenberns gmail com>
23 Oct
to Gerd, Francesco, marmuta
Hi Gerd,
On 23 October 2013 09:20, Gerd Kohlberger <gerdko gmail com> wrote:
Hi Magdalen,
yeah I didn't see your mail until now. Sorry. Francesco and I
haven't really talked about the future of mousetweaks since your gsoc
mails. (congratulations by the way)
Thanks!
I think porting mousetweaks in its current form to wayland is not
going to work. To implement hover-click we would need 2 things: a) get
global mouse-motion events and b) synthesize/fake mouse button events. A
regular application can do neither under wayland. I can think of 2 ways
to solve this, either move everything into gnome-shell as you suggest,
or keep mousetweaks as a standalone daemon and use at-spi.
atspi is now in GNOME Shell too but there are problems because it is
making synchronous calls at the moment.
at-spi has API for global mouse-events (AtspiEventListener +
"mouse:*" events) and for faking mouse events
(atspi_generate_mouse_event()), but both operations don't work under
wayland at the moment. Until those things are implemented we can't start
our port.
Reimplementing mousetweaks in gnome-shell would have a lot of
advantages. It would be easy to get mouse-events and the click-selection
window could be integrated directly into the chrome. It's currently
impossible to drag things around in the Activites overview, because you
can't change the hover-click type from within the overview. Another
advantage would be that we save a lot of code. Probably 90% of
mousetweaks only deals with being a daemon and getting various events,
the actual hover-click logic is only a tiny part.
I will try this out.
There is however one problem, if all the features move into
gnome-shell then mousetweaks becomes a GNOME-only solution. At the
moment mousetweaks works fine on Unity, lxde and even KDE. It can also
be used as a service. For example the default on-screen keyboard on
Ubuntu (onboard) integrates mousetweaks into its interface. You can
activate hover-click and select the different click-types directly with
the keyboard.
This might be moving to gnome-shell as well. There was talk of it in
GUADEC but I don't know what was decided.
I would actually prefer to just start working on a hoverclick.js
component for gnome-shell (or you could Magdalen if you like), but
loosing the onboard integration is a big problem. Francesco, what do you
think? I'm guessing you're not even running gnome-shell.
Me too.
Cheers.
CC'ing marmuta (main onboard developer)
marmuta, what do you think about droping mousetweaks and also
implementing hover-click? The at-spi dependency is already there and it
might even be easier to do it ourselves than to do ipc with mousetweaks.
It would at least resolve the issues with hiding the click-selection
window.
Can I cc Alejandro into this thread as well?
Francesco Fumanti
23 Oct
to Gerd, me, marmuta
Hi Magdalen,
I am pleased to read that you are still interested in the accessibility
features, and particularly in mousetweaks.
On 2013-10-23 10:20, Gerd Kohlberger wrote:
Hi Magdalen,
yeah I didn't see your mail until now. Sorry. Francesco and I
haven't really talked about the future of mousetweaks since your gsoc
mails. (congratulations by the way)
I think porting mousetweaks in its current form to wayland is not
going to work. To implement hover-click we would need 2 things: a) get
global mouse-motion events and b) synthesize/fake mouse button events. A
regular application can do neither under wayland. I can think of 2 ways
to solve this, either move everything into gnome-shell as you suggest,
or keep mousetweaks as a standalone daemon and use at-spi.
at-spi has API for global mouse-events (AtspiEventListener +
"mouse:*" events) and for faking mouse events
(atspi_generate_mouse_event()), but both operations don't work under
wayland at the moment. Until those things are implemented we can't start
our port.
Reimplementing mousetweaks in gnome-shell would have a lot of
advantages. It would be easy to get mouse-events and the click-selection
window could be integrated directly into the chrome. It's currently
impossible to drag things around in the Activites overview, because you
can't change the hover-click type from within the overview. Another
advantage would be that we save a lot of code. Probably 90% of
mousetweaks only deals with being a daemon and getting various events,
the actual hover-click logic is only a tiny part.
There is however one problem, if all the features move into
gnome-shell then mousetweaks becomes a GNOME-only solution. At the
moment mousetweaks works fine on Unity, lxde and even KDE. It can also
be used as a service. For example the default on-screen keyboard on
Ubuntu (onboard) integrates mousetweaks into its interface. You can
activate hover-click and select the different click-types directly with
the keyboard.
I would actually prefer to just start working on a hoverclick.js
component for gnome-shell (or you could Magdalen if you like), but
loosing the onboard integration is a big problem. Francesco, what do you
think? I'm guessing you're not even running gnome-shell.
You are right: I have not been running gnome-shell for many months.
The advantages of going GNOME-only seem to be primarily from the point
of view of the implementation. The dwell user would get tied to
gnome-shell unless each desktop environment implements its own click
accessibility features. This would be a clear regression for people
relying on these features compared to the current situation.
Assuming Wayland finally offers the things that the mousetweaks daemon
needs to be able to work. Could mousetweaks not be enhanced that it
checks which display server is running and run for that particular
server if it is already supported by mousetweaks? (I suppose that MIR
might have also to be considered in the near future.) The different
desktop environments could (continue to) use the daemon similarly to how
Onboard is doing it.
Am I missing something, when I assume that gnome-shell could offer some
sort of GUI that uses the mousetweaks daemon as a service?
Going the at-spi2 way: The first incarnation of mousetweaks used the old
at-spi until it was ported to X to remove the at-spi dependency. at-spi2
is a clear improvement in confront to the old at-spi. I see the
advantage of using at-spi2, as (I assume) the daemon does not have to be
adapted to the display servers. The code will probably be simpler and
easier to maintain. But, what are the drawbacks compared to a daemon
based on the display server? I don't have the necessary know-how to
evaluate it, but I wonder whether it will work for example with all the
toolkits.
CC'ing marmuta (main onboard developer)
marmuta, what do you think about droping mousetweaks and also
implementing hover-click? The at-spi dependency is already there and it
might even be easier to do it ourselves than to do ipc with mousetweaks.
It would at least resolve the issues with hiding the click-selection
window.
Gerd, does this mean that the implementation of the ipc is more
complicated than the implementation of the click features by using
at-spi2?
Moreover, this leads to the question of whether a mousetweaks based on
at-spi2 makes sense at all, unless we want to avoid that everybody
implements its own version of the click accessibility features.
Cheers,
Francesco
PS: Gerd, could you please add some pieces of information to the
mousetweaks section of the following page?
https://wiki.gnome.org/Accessibility/Wayland
marmuta
23 Oct
to Francesco, Gerd, me
Technically we probably could, at least until we move on with Onboard
too and start targeting Wayland and/or MIR. I can't tell yet how much of
this would survive the porting, same for all the other X tricks in
Onboard.
Assuming we did it, how would that work with Onboard in gnome-shell?
There are two competing hover-click providers then, would we enable the
gnome way in gnome-shell and fall-back to Onboard's implementation
anywhere else? Wouldn't we then still need to be able to communicate
with the new shell-mousetweaks?
Magdalen Berns <magdalenberns gmail com>
24 Oct
to marmuta, Francesco, Gerd
Hi all,
A problem we have is that mousetweaks in its current form won't work
under Wayland. Or at least I do not think so based on what I am finding
out about it. The easiest way to make it work in gnome is to implement
it into GNOME Shell but I do not know whether that could/should mean the
standalone is abandoned altogether. If so that is not ideal.
I do not see a reason to abandon mousetweaks as a standalone (other than
lack of time and resources) but I do not know enough about the what work
(if any) is currently needed for mousetweaks to continue to work in as a
standalone outside of wayland. What can you all tell me about that?
Kind regards,
Magdalen
marmuta
25 Oct
to me, Francesco, Gerd
Hi,
A problem we have is that mousetweaks in its current form won't work
under Wayland. Or at least I do not think so based on what I am
finding out about it.
I'm wondering, is there any way to implement mousetweaks at the evdev
level? Events could probably be generated with /dev/uinput.
The easiest way to make it work in gnome is to
implement it into GNOME Shell but I do not know whether that
could/should mean the standalone is abandoned altogether. If so that
is not ideal.
I do not see a reason to abandon mousetweaks as a standalone (other
than lack of time and resources) but I do not know enough about the
what work (if any) is currently needed for mousetweaks to continue to
work in as a standalone outside of wayland. What can you all tell me
about that?
Only Gerd can answer that, but I guess once the main desktops switched
away from X, there will be few incentives to keep maintaining it.
Moving parts of it to Onboard then, as he suggested, may make sense,
because for many desktops there is no other convenient way to enable
mousetweaks anyway.
Here's a thought, though: how about keeping standalone mousetweaks as a
cross-desktop unified interface. It would provide the old functionality
for X-based desktops, but only act as a proxy where desktop specific
implementations are available, e.g. the shell implementation in GNOME
shell. Would that make sense?
Kind regards,
Magdalen
Best regards, M.
Gerd Kohlberger
25 Oct
to me, Francesco, marmuta
On 23.10.2013 17:16, Magdalen Berns wrote:
Hi Gerd,
On 23 October 2013 09:20, Gerd Kohlberger <gerdko gmail com
<mailto:gerdko gmail com>> wrote:
Hi Magdalen,
yeah I didn't see your mail until now. Sorry. Francesco and I
haven't really talked
about the future of mousetweaks since your gsoc mails.
(congratulations by the way)
Thanks!
I think porting mousetweaks in its current form to wayland is
not going to work. To
implement hover-click we would need 2 things: a) get global
mouse-motion events and b)
synthesize/fake mouse button events. A regular application can
do neither under
wayland. I can think of 2 ways to solve this, either move
everything into gnome-shell
as you suggest, or keep mousetweaks as a standalone daemon and
use at-spi.
atspi is now in GNOME Shell too but there are problems because it
is making synchronous
calls at the moment.
at-spi has API for global mouse-events (AtspiEventListener +
"mouse:*" events) and for
faking mouse events (atspi_generate_mouse_event())__, but both
operations don't work
under wayland at the moment. Until those things are implemented
we can't start our port.
Reimplementing mousetweaks in gnome-shell would have a lot of
advantages. It would be
easy to get mouse-events and the click-selection window could be
integrated directly
into the chrome. It's currently impossible to drag things around
in the Activites
overview, because you can't change the hover-click type from
within the overview.
Another advantage would be that we save a lot of code. Probably
90% of mousetweaks
only deals with being a daemon and getting various events, the
actual hover-click
logic is only a tiny part.
I will try this out.
I found 2 things that don't work in the overview. You can't move windows
between viewports and you can't rearrange the icons in the launcher.
Both things aren't show-stoppers but it would be nice to have everything
work.
There is however one problem, if all the features move into
gnome-shell then
mousetweaks becomes a GNOME-only solution. At the moment
mousetweaks works fine on
Unity, lxde and even KDE. It can also be used as a service. For
example the default
on-screen keyboard on Ubuntu (onboard) integrates mousetweaks
into its interface. You
can activate hover-click and select the different click-types
directly with the keyboard.
This might be moving to gnome-shell as well. There was talk of it in
GUADEC but I don't
know what was decided.
Do you mean hover-click controls in caribou? This could solve the
overview problem because caribou can pop up there.
I would actually prefer to just start working on a hoverclick.js
component for
gnome-shell (or you could Magdalen if you like), but loosing the
onboard integration
is a big problem. Francesco, what do you think? I'm guessing
you're not even running
gnome-shell.
Me too.
Cheers.
CC'ing marmuta (main onboard developer)
marmuta, what do you think about droping mousetweaks and also
implementing
hover-click? The at-spi dependency is already there and it might
even be easier to do
it ourselves than to do ipc with mousetweaks. It would at least
resolve the issues
with hiding the click-selection window.
Can I cc Alejandro into this thread as well?
Sure. Just cc people you think might be interested.
Cheers.
Gerd Kohlberger
25 Oct
to marmuta, Francesco, me
Yes that's my main concern. Most mousetweaks users I've had contact with
were using Ubuntu and probably Unity.
The advantages of going GNOME-only seem to be primarily from the
point of view of the implementation. The dwell user would get
tied to
gnome-shell unless each desktop environment implements its own
click
accessibility features. This would be a clear regression for
people
relying on these features compared to the current situation.
Yes.
Assuming Wayland finally offers the things that the mousetweaks
daemon needs to be able to work. Could mousetweaks not be
enhanced
that it checks which display server is running and run for that
particular server if it is already supported by mousetweaks? (I
suppose that MIR might have also to be considered in the near
future.) The different desktop environments could (continue to)
use
the daemon similarly to how Onboard is doing it.
If we only depend on atspi and not use any display server specific
functions then we don't need different backends. The backends would be
in the atspi registry. I don't know if there are any plans for a MIR
backend in atspi.
Am I missing something, when I assume that gnome-shell could
offer some sort of
GUI that uses the mousetweaks daemon as a service?
No that would still be possible.
Going the at-spi2 way: The first incarnation of mousetweaks used
the
old at-spi until it was ported to X to remove the at-spi
dependency.
at-spi2 is a clear improvement in confront to the old at-spi. I
see
the advantage of using at-spi2, as (I assume) the daemon does
not
have to be adapted to the display servers. The code will
probably be
simpler and easier to maintain. But, what are the drawbacks
compared
to a daemon based on the display server? I don't have the
necessary
know-how to evaluate it, but I wonder whether it will work for
example with all the toolkits.
The mouse-event parts of atspi are independent from the toolkit support
so that shouldn't be a problem.
I can think of 2 regressions. The cursor animations are X specific and
have to be removed. I don't think it's possible to implement them under
wayland or mir. It could work from within gnome-shell, the shell
magnifier also manipulates the cursor sprite. But I also wouldn't mind
loosing this feature, it never worked reliably even on X. I'm curious,
do any of you see the cursor animations? When hover-click is activated
the mouse cursor should fill up with a color before a click happens.
The other thing are mouse wheel events. Currently mousetweaks stops the
hover-click timer if the wheel is used. eg. if you scroll down on a web
page with the wheel you don't want mousetweaks to randomly click
somewhere in between. atspi only supports button 1-3.
CC'ing marmuta (main onboard developer)
marmuta, what do you think about droping mousetweaks and
also
implementing hover-click? The at-spi dependency is already
there
and it might even be easier to do it ourselves than to do
ipc with
mousetweaks. It would at least resolve the issues with
hiding the
click-selection window.
Technically we probably could, at least until we move on with
Onboard
too and start targeting Wayland and/or MIR. I can't tell yet how
much of
this would survive the porting, same for all the other X tricks in
Onboard.
Onboard without xkb, xinput and xtest sounds almost impossible.
Assuming we did it, how would that work with Onboard in gnome-shell?
There are two competing hover-click providers then, would we enable
the
gnome way in gnome-shell and fall-back to Onboard's implementation
anywhere else? Wouldn't we then still need to be able to communicate
with the new shell-mousetweaks?
I didn't think of that. Maybe always use onboard's and disable the other
one, but that could get complicated fast, particularly if there are
multiple other implementations.
Gerd, does this mean that the implementation of the ipc is more
complicated than the implementation of the click features by
using
at-spi2?
hover-click is really just a timer. If the timer finishes we send a fake
click, if the mouse is moved before that, we reset the timer and start
again. If an app can already get global mouse events and synthesize
clicks then this requires only a couple lines of code.
Moreover, this leads to the question of whether a mousetweaks
based
on at-spi2 makes sense at all, unless we want to avoid that
everybody
implements its own version of the click accessibility features.
Yes, but what I fear will happen is that nobody implements them. We can
work on shell and onboard, but it would disappear everywhere else. With
Canonical's CLA I certainly won't contribute anything to Ubuntu.
Cheers.
Gerd Kohlberger
25 Oct
to marmuta, me, Francesco
Hi,
On 25.10.2013 02:17, marmuta wrote:
Hi,
A problem we have is that mousetweaks in its current form won't
work
under Wayland. Or at least I do not think so based on what I am
finding out about it.
I'm wondering, is there any way to implement mousetweaks at the
evdev
level? Events could probably be generated with /dev/uinput.
evdev and uinput was my first thought as well. Unfortunately all the
files are root:root, mousetweaks would have to be root or use some sort
of suid helper. It's also complicated to use evdev directly, you need
udev for PNP, figure out the device types yourself and you have to
disconnect from all devices once a VT switch occurs. (requires talking
to the DRM, again root only) Examples for that are in weston-launch.c
and the weston evdev files.
The easiest way to make it work in gnome is to
implement it into GNOME Shell but I do not know whether that
could/should mean the standalone is abandoned altogether. If so
that
is not ideal.
I do not see a reason to abandon mousetweaks as a standalone
(other
than lack of time and resources) but I do not know enough about
the
what work (if any) is currently needed for mousetweaks to
continue to
work in as a standalone outside of wayland. What can you all
tell me
about that?
mousetweaks uses a lot of X specific function calls throughout the code.
All of those would have to be replaced with at-spi equivalents. Once
this is done mousetweaks should work everywhere at-spi does, including
wayland and maybe even MIR in the future. I'll start working on a port
to atspi over the weekend to see if there are any unexpected problems.
Only Gerd can answer that, but I guess once the main desktops
switched
away from X, there will be few incentives to keep maintaining it.
Moving parts of it to Onboard then, as he suggested, may make sense,
because for many desktops there is no other convenient way to enable
mousetweaks anyway.
Here's a thought, though: how about keeping standalone mousetweaks
as a
cross-desktop unified interface. It would provide the old
functionality
for X-based desktops, but only act as a proxy where desktop specific
implementations are available, e.g. the shell implementation in
GNOME
shell. Would that make sense?
Wouldn't this require that mousetweaks is always installed? It's
installed by default on Ubuntu but I don't know what other distributions
do. In a way gnome-settings-daemon already acts as that proxy. It
monitors the 'dwell-click-enabled' key and launches mousetweaks. Maybe
it could instead check if gnome-shell is running and use mousetweaks
only as a fallback.
Cheers.
Kind regards,
Magdalen
Best regards, M.
Francesco Fumanti
25 Oct
to Gerd, marmuta, me
Hi,
I am probably wrong, but I interpreted marmuta's remark more broadly:
By standalone mousetweaks, I understood a package containing the daemon
and a standalone gui application that could start and control the daemon
by itself; for example over dbus (I am assuming here, that the daemon
can be started over dbus).
Thus, people on distributions not offering mousetweaks, could install
the package and make use of it through the provided gui application. And
distributions wanting a better integration, could only use the daemon
and offer their own graphical interface.
Cheers,
Francesco
Magdalen Berns <magdalenberns gmail com>
25 Oct
to Piñeiro, Francesco, Gerd, marmuta
cc Alejandro
On 25 October 2013 13:59, Francesco Fumanti <francesco fumanti gmx net>
wrote:
Hi,
On 2013-10-25 13:55, Gerd Kohlberger wrote:
A problem we have is that mousetweaks in its current
form won't work
under Wayland. Or at least I do not think so based on
what I am
finding out about it.
I'm wondering, is there any way to implement mousetweaks at
the evdev
level? Events could probably be generated with /dev/uinput.
evdev and uinput was my first thought as well. Unfortunately all
the files are root:root, mousetweaks would have to be root or use some
sort of suid helper. It's also complicated to use evdev directly, you
need udev for PNP, figure out the device types yourself and you have to
disconnect from all devices once a VT switch occurs. (requires talking
to the DRM, again root only) Examples for that are in weston-launch.c
and the weston evdev files.
The easiest way to make it work in gnome is to
implement it into GNOME Shell but I do not know whether
that
could/should mean the standalone is abandoned
altogether. If so that
is not ideal.
I do not see a reason to abandon mousetweaks as a
standalone (other
than lack of time and resources) but I do not know
enough about the
what work (if any) is currently needed for mousetweaks
to continue to
work in as a standalone outside of wayland. What can you
all tell me
about that?
mousetweaks uses a lot of X specific function calls throughout
the code. All of those would have to be replaced with at-spi
equivalents. Once this is done mousetweaks should work everywhere at-spi
does, including wayland and maybe even MIR in the future. I'll start
working on a port to atspi over the weekend to see if there are any
unexpected problems.
Only Gerd can answer that, but I guess once the main
desktops switched
away from X, there will be few incentives to keep
maintaining it.
Moving parts of it to Onboard then, as he suggested, may
make sense,
because for many desktops there is no other convenient way
to enable
mousetweaks anyway.
Here's a thought, though: how about keeping standalone
mousetweaks as a
cross-desktop unified interface. It would provide the old
functionality
for X-based desktops, but only act as a proxy where desktop
specific
implementations are available, e.g. the shell implementation
in GNOME
shell. Would that make sense?
Wouldn't this require that mousetweaks is always installed? It's
installed by default on Ubuntu but I don't know what other distributions
do. In a way gnome-settings-daemon already acts as that proxy. It
monitors the 'dwell-click-enabled' key and launches mousetweaks. Maybe
it could instead check if gnome-shell is running and use mousetweaks
only as a fallback.
I am probably wrong, but I interpreted marmuta's remark more
broadly:
By standalone mousetweaks, I understood a package containing the
daemon and a standalone gui application that could start and control the
daemon by itself; for example over dbus (I am assuming here, that the
daemon can be started over dbus).
Thus, people on distributions not offering mousetweaks, could
install the package and make use of it through the provided gui
application. And distributions wanting a better integration, could only
use the daemon and offer their own graphical interface.
Cheers,
Francesco
Piñeiro
25 Oct
to jdiggs, me, Francesco, Gerd, marmuta
On 10/25/2013 05:18 PM, Magdalen Berns wrote:
cc Alejandro
Hi, thanks for the forward.
I have some comments, suggestions below.
On 25 October 2013 13:59, Francesco Fumanti
<francesco fumanti gmx net> wrote:
Hi,
On 2013-10-25 13:55, Gerd Kohlberger wrote:
A problem we have is that mousetweaks in its current
form won't work
under Wayland. Or at least I do not think so based on
what I am
finding out about it.
I'm wondering, is there any way to implement mousetweaks
at the evdev
level? Events could probably be generated
with /dev/uinput.
evdev and uinput was my first thought as well. Unfortunately
all the files are root:root, mousetweaks would have to be root or use
some sort of suid helper. It's also complicated to use evdev directly,
you need udev for PNP, figure out the device types yourself and you have
to disconnect from all devices once a VT switch occurs. (requires
talking to the DRM, again root only) Examples for that are in
weston-launch.c and the weston evdev files.
Well, although I don't have a lot of experience with Wayland, the
feeling that I have is that the main idea is Wayland being more simple
that X, and moving a lot of the functionality to the clients (example:
window decorators now is a responsibilit of the client), but at the same
time right now is really complex from the client side to implement some
of the stuff that was possible before with X. As far as I understand
that paragrah, before X exposed that functionality, so clients didn't
have to talk directly to evdev/uinput, and now is not possible.
The easiest way to make it work in gnome is to
implement it into GNOME Shell but I do not know
whether that
could/should mean the standalone is abandoned
altogether. If so that
is not ideal.
I do not see a reason to abandon mousetweaks as a
standalone (other
than lack of time and resources) but I do not know
enough about the
what work (if any) is currently needed for mousetweaks
to continue to
work in as a standalone outside of wayland. What can
you all tell me
about that?
mousetweaks uses a lot of X specific function calls throughout
the code. All of those would have to be replaced with at-spi
equivalents. Once this is done mousetweaks should work everywhere at-spi
does, including wayland and maybe even MIR in the future. I'll start
working on a port to atspi over the weekend to see if there are any
unexpected problems.
Well, that idea makes sense in the theory, but there are some problems:
* On Wayland: there are still several questions about how to implement
some of the current API on Wayland. For example, you mentioned some
problems using evdev/uinput on mousetweaks. I foresee a similar problem
with at-spi (that also runs as a user process).
* On X: one of the outcomes of investigating how to reimplement some
of the current functionality of at-spi2 on Wayland, is the conclusion
that the usage of X on at-spi is obsolete. Needs and update, and
probably some of the current functionality is not working.
* at-spi2 itself: other of the outcomes of that investigation is
realizing that at-spi2 API needs an update. Current API is confusing.
Hopefully can be partially solved improving the documentation, but
likely, it would be required to tweak the API itself. Any feedback would
be welcome.
Finally, FWIW, there are some running threads about Wayland and
accessibility on gnome-accessibility-devel list and on the wayland-devel
list itself [1]. Probably it would be good to collaborate publicly on
those discussions, and see what the people with more experience on
Wayland thinks.
Thanks for all this information
Best regards
[1]
http://lists.freedesktop.org/archives/wayland-devel/2013-October/011487.html
marmuta
26 Oct
to Gerd, Francesco, me
I don't see the animation for the default arrow, but all other cursors,
caret, resizing, etc. fill up with what looks like the orange selection
color.
The other thing are mouse wheel events. Currently mousetweaks stops
the hover-click timer if the wheel is used. eg. if you scroll down on
a web page with the wheel you don't want mousetweaks to randomly
click somewhere in between. atspi only supports button 1-3.
CC'ing marmuta (main onboard developer)
marmuta, what do you think about droping mousetweaks and also
implementing hover-click? The at-spi dependency is already there
and it might even be easier to do it ourselves than to do ipc with
mousetweaks. It would at least resolve the issues with hiding the
click-selection window.
Technically we probably could, at least until we move on with
Onboard too and start targeting Wayland and/or MIR. I can't tell
yet how much of this would survive the porting, same for all the
other X tricks in Onboard.
Onboard without xkb, xinput and xtest sounds almost impossible.
It won't be easy, I agree, but I suspect there will be alternatives, no
way there won't be a replacement for xkb.
xinput might not be essential, scanner and click generation could
perhaps switch to GdkDevice.
xtest could perhaps be replaced with uinput (daemon?). I have an old
uinput branch of Onboard that kinda worked. If a rudimentary
osk_uinput.c would help you anything with mousetweaks let me know.
Else we could revisit at-spi2 for generating key-strokes, but there's no
consistent separation between press and release, unfortunately.
All the GUI stuff will hopefully just work, I had it start up in Weston
a year ago or so, and it looked alright.
Assuming we did it, how would that work with Onboard in gnome-shell?
There are two competing hover-click providers then, would we enable
the gnome way in gnome-shell and fall-back to Onboard's
implementation anywhere else? Wouldn't we then still need to be
able to communicate with the new shell-mousetweaks?
I didn't think of that. Maybe always use onboard's and disable the
other one, but that could get complicated fast, particularly if there
are multiple other implementations.
I think, if there indeed will be multiple implementations, we better
yield to them in Onboard. No collisions that way and Onboard wouldn't
have to deal with the platform's intricacies. Activities view and the
locked down password dialog in GS come to mind.
Gerd, does this mean that the implementation of the ipc is more
complicated than the implementation of the click features by using
at-spi2?
hover-click is really just a timer. If the timer finishes we send a
fake click, if the mouse is moved before that, we reset the timer and
start again. If an app can already get global mouse events and
synthesize clicks then this requires only a couple lines of code.
Moreover, this leads to the question of whether a mousetweaks based
on at-spi2 makes sense at all, unless we want to avoid that
everybody implements its own version of the click accessibility
features.
Yes, but what I fear will happen is that nobody implements them. We
can work on shell and onboard, but it would disappear everywhere
else. With Canonical's CLA I certainly won't contribute anything to
Ubuntu.
Cheers.
Cheers, M.
Gerd Kohlberger
28 Oct
to marmuta, Francesco, me
Thanks. It's the same here.
The other thing are mouse wheel events. Currently mousetweaks
stops
the hover-click timer if the wheel is used. eg. if you scroll
down on
a web page with the wheel you don't want mousetweaks to randomly
click somewhere in between. atspi only supports button 1-3.
CC'ing marmuta (main onboard developer)
marmuta, what do you think about droping mousetweaks
and also
implementing hover-click? The at-spi dependency is
already there
and it might even be easier to do it ourselves than
to do ipc with
mousetweaks. It would at least resolve the issues
with hiding the
click-selection window.
Technically we probably could, at least until we move on
with
Onboard too and start targeting Wayland and/or MIR. I can't
tell
yet how much of this would survive the porting, same for all
the
other X tricks in Onboard.
Onboard without xkb, xinput and xtest sounds almost impossible.
It won't be easy, I agree, but I suspect there will be alternatives,
no
way there won't be a replacement for xkb.
xinput might not be essential, scanner and click generation could
perhaps switch to GdkDevice.
xtest could perhaps be replaced with uinput (daemon?). I have an old
uinput branch of Onboard that kinda worked. If a rudimentary
osk_uinput.c would help you anything with mousetweaks let me know.
How did you open fd? I'm getting
gerd box01:~$ ll /dev/uinput
crw------- 1 root root 10, 223 Okt 21 17:34 /dev/uinput
Else we could revisit at-spi2 for generating key-strokes, but
there's no
consistent separation between press and release, unfortunately.
All the GUI stuff will hopefully just work, I had it start up in
Weston
a year ago or so, and it looked alright.
Assuming we did it, how would that work with Onboard in
gnome-shell?
There are two competing hover-click providers then, would we
enable
the gnome way in gnome-shell and fall-back to Onboard's
implementation anywhere else? Wouldn't we then still need to
be
able to communicate with the new shell-mousetweaks?
I didn't think of that. Maybe always use onboard's and disable
the
other one, but that could get complicated fast, particularly if
there
are multiple other implementations.
I think, if there indeed will be multiple implementations, we better
yield to them in Onboard. No collisions that way and Onboard
wouldn't
have to deal with the platform's intricacies. Activities view and
the
locked down password dialog in GS come to mind.
ok.
Gerd Kohlberger
28 Oct
to Piñeiro, me, Francesco, marmuta, jdiggs
Hi Alejandro,
On 25.10.2013 18:24, Piñeiro wrote:
On 10/25/2013 05:18 PM, Magdalen Berns wrote:
cc Alejandro
Hi, thanks for the forward.
I have some comments, suggestions below.
On 25 October 2013 13:59, Francesco Fumanti
<francesco fumanti gmx net
<mailto:francesco fumanti gmx net>> wrote:
Hi,
On 2013-10-25 13:55, Gerd Kohlberger wrote:
A problem we have is that mousetweaks in its
current form won't work
under Wayland. Or at least I do not think so
based on what I am
finding out about it.
I'm wondering, is there any way to implement
mousetweaks at the evdev
level? Events could probably be generated
with /dev/uinput.
evdev and uinput was my first thought as well.
Unfortunately all the files are
root:root, mousetweaks would have to be root or use some
sort of suid helper.
It's also complicated to use evdev directly, you need
udev for PNP, figure out
the device types yourself and you have to disconnect
from all devices once a VT
switch occurs. (requires talking to the DRM, again root
only) Examples for that
are in weston-launch.c and the weston evdev files.
Well, although I don't have a lot of experience with Wayland, the
feeling that I have is
that the main idea is Wayland being more simple that X, and moving a
lot of the
functionality to the clients (example: window decorators now is a
responsibilit of the
client), but at the same time right now is really complex from the
client side to
implement some of the stuff that was possible before with X. As far
as I understand that
paragrah, before X exposed that functionality, so clients didn't
have to talk directly to
evdev/uinput, and now is not possible.
On X the XTest extension makes it easy to synthesize mouse events and
you can poll for the mouse position and button state. The code in
mousetweaks to do that is exactly the same as in the at-spi-registryd.
At the time we didn't want to use at-spi because you had to do the
enable a11y/restart the session dance and it seemed better to just
rewrite those parts. But now that a11y is always-on it's just code
duplication.
The easiest way to make it work in gnome is to
implement it into GNOME Shell but I do not know
whether that
could/should mean the standalone is abandoned
altogether. If so that
is not ideal.
I do not see a reason to abandon mousetweaks as
a standalone (other
than lack of time and resources) but I do not
know enough about the
what work (if any) is currently needed for
mousetweaks to continue to
work in as a standalone outside of wayland. What
can you all tell me
about that?
mousetweaks uses a lot of X specific function calls
throughout the code. All of
those would have to be replaced with at-spi equivalents.
Once this is done
mousetweaks should work everywhere at-spi does,
including wayland and maybe even
MIR in the future. I'll start working on a port to atspi
over the weekend to see
if there are any unexpected problems.
Well, that idea makes sense in the theory, but there are some
problems:
* On Wayland: there are still several questions about how to
implement some of the
current API on Wayland. For example, you mentioned some problems
using evdev/uinput on
mousetweaks. I foresee a similar problem with at-spi (that also runs
as a user process).
* On X: one of the outcomes of investigating how to reimplement
some of the current
functionality of at-spi2 on Wayland, is the conclusion that the
usage of X on at-spi is
obsolete. Needs and update, and probably some of the current
functionality is not working.
mousetweaks only uses 2 at-spi functions, the generateMouseEvent method
and an EventListener to listen for "mouse:*" events. Does this mean that
those things will no longer work on X once wayland support is added?
I've now uploaded a new 'wayland' branch to git. On X everything seems
to work as before but on wayland mousetweaks just sits there and waits
for at-spi events that never arrive. And I've also updated the wayland
accessibility page. Hope that's ok.
https://git.gnome.org/browse/mousetweaks/tree/?h=wayland
https://wiki.gnome.org/Accessibility/Wayland
* at-spi2 itself: other of the outcomes of that investigation is
realizing that at-spi2
API needs an update. Current API is confusing. Hopefully can be
partially solved improving
the documentation, but likely, it would be required to tweak the API
itself. Any feedback
would be welcome.
Finally, FWIW, there are some running threads about Wayland and
accessibility on
gnome-accessibility-devel list and on the wayland-devel list itself
[1]. Probably it would
be good to collaborate publicly on those discussions, and see what
the people with more
experience on Wayland thinks.
Ok, thanks for the info.
Cheers,
Gerd
marmuta
28 Oct
to Gerd, Francesco, me
I probably just changed permissions to 666. That code was for
investigating lost key-strokes back then, not really meant to be
shipped. However, when cornered in Wayland/MIR, with uinput the only
way out, I would consider building a uinput daemon that opens the
device as root.
Piñeiro
28 Oct
to Gerd, me, Francesco, marmuta, jdiggs
Hi,
On 10/28/2013 11:35 AM, Gerd Kohlberger wrote:
Hi Alejandro,
Well, although I don't have a lot of experience with Wayland, the
feeling that I have is
that the main idea is Wayland being more simple that X, and moving a
lot of the
functionality to the clients (example: window decorators now is a
responsibilit of the
client), but at the same time right now is really complex from the
client side to
implement some of the stuff that was possible before with X. As far
as I understand that
paragrah, before X exposed that functionality, so clients didn't have
to talk directly to
evdev/uinput, and now is not possible.
On X the XTest extension makes it easy to synthesize mouse events and
you can poll for the mouse position and button state. The code in
mousetweaks to do that is exactly the same as in the at-spi-registryd.
At the time we didn't want to use at-spi because you had to do the
enable a11y/restart the session dance and it seemed better to just
rewrite those parts. But now that a11y is always-on it's just code
duplication.
During GUADEC, Keith Packard, a X developer, attended the accessibility
BOF. He mentioned that using XTest to synthesize mouse events is ok, but
not the best for tracking mouse position. Basically due the need of
polling. He mentioned that it would be better to use recent additions to
X, specifically xinput2. He event bloggued about that:
http://keithp.com/blogs/Cursor_tracking/
Anyway, it is true that if mousetweaks were doing the same that at-spi2,
I agree that just using at-spi2 would be better, to avoid replicate
code.
Well, that idea makes sense in the theory, but there are some
problems:
* On Wayland: there are still several questions about how to
implement some of the
current API on Wayland. For example, you mentioned some problems
using evdev/uinput on
mousetweaks. I foresee a similar problem with at-spi (that also runs
as a user process).
* On X: one of the outcomes of investigating how to reimplement
some of the current
functionality of at-spi2 on Wayland, is the conclusion that the usage
of X on at-spi is
obsolete. Needs and update, and probably some of the current
functionality is not working.
mousetweaks only uses 2 at-spi functions, the generateMouseEvent
method and an EventListener to listen for "mouse:*" events. Does this
mean that those things will no longer work on X once wayland support
is added?
Sorry if I didn't explain myself properly. What I mentioned about X
doesn't have any relation with adding wayland. What I wanted to say is
that some of the code is using somewhat old X methods that it would be
to replace (ie, with some xinput2 stuff). But we don't foresee any
regression there.
I've now uploaded a new 'wayland' branch to git. On X everything seems
to work as before
Good to know. As I said, although ideally we would like to update the
code that depends on X, with the proper X methods, we will not do that
if that means a regression. If we do that, we will ping you.
but on wayland mousetweaks just sits there and waits for at-spi events
that never arrive.
Well, the problem is that on at-spi2 there are not a equivalent
implementation relying on wayland. Basically, we are on a situation of
not being sure how to do that using just wayland methods, with something
similar to what you commented on previous emails.
And I've also updated the wayland accessibility page. Hope that's ok.
Yes it is ok, and in fact, welcomed. Thanks for the update.
BR
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]
