Re: [orca-list] Odd issue with the Riot web client

[Nolan Darilek <nolan thewordnerd info>]

Think I had a possible breakthrough on this one. The problem went away 
for a while, so I thought it was fixed in Orca. Then it came back, and 
was always reproduceable.


I discovered that my wireless mouse was off when the issue was always 
reproduceable. With the mouse on, and stashed out of the way so it 
doesn't get accidentally bumped, the issue becomes significantly more 
manageable. I'm hesitant to say it goes away completely, but riot.im 
becomes much more usable.


I was also getting odd behavior in the Electron desktop client, which 
may still be on an older Electron. Focus wouldn't move despite me 
arrowing up. This, too, seems to have gone away when I turned my mouse 
back on.


Turning it off again makes things get odd again, but not in the same 
way. I don't seem to get messages read when arrowing around having 
turned off my mouse. So there appear to be three distinct states:


1. Mouse is off, and was never turned on after boot. I'm wondering if 
this never produces a mouse pointer, and if this lack of a mouse pointer 
makes Firefox, X, or Orca unhappy.


2. Mouse gets turned on. Things behave significantly better when this 
happens.


3. Mouse gets turned back off. I wonder if, having gotten mouse pointer 
input at some point, some component of the system retains that last 
state and either leaves a mouse pointer at its previous location, or 
just doesn't clear some bit of state. Things behave oddly here too, but 
not in a way I can easily quantify.


So, in short, if my mouse is never turned on post-boot, something 
doesn't work either in Riot Firefox, or Riot Desktop/Electron. Glad I 
found a workaround, and hope this helps us figure this out.

On 1/23/20 7:53 AM, Joanmarie Diggs wrote:
> Hey again Nolan.
>
> Update: I found a way to reproduce this, but it's only broken in 
> Firefox. Orca + Chromium (at least my locally-built chromium) works as 
> expected. Toolbar goes poof and the press of Up Arrow moves you to the 
> message above the one which went poof. So.... I'll add this to my todo 
> list.
>
> If you're curious, what I suspect is be happening is that your mouse 
> pointer just so happens to be over a chat message. As a result, the 
> message it just so happens to be over has an associated 
> message-actions toolbar which you are able to arrow to (though I wind 
> up on the toolbar; not a button child). Then something is causing the 
> toolbar you're in to vanish. This can happen if you move the physical 
> mouse pointer or if the associated message scrolls off screen (e.g. as 
> a side effect of your arrowing).
>
> --joanie
>
> On 1/22/20 7:14 PM, Joanmarie Diggs wrote:
> > Hey Nolan.
> >
> > tl;dr: Seems that you're in the Message Actions toolbar when it goes 
> > poof! The document itself then claims focus. Orca tries to figure out 
> > where you should be. It concludes the list (good), but the first item 
> > in that list (bad). I can try to improve things in Orca -- which 
> > already has a lot of logic to recover from things going poof! But 
> > maybe the Riot devs can help with clues and/or solutions on their 
> > end? Also, push buttons should have names. At least one in Riot 
> > apparently doesn't....
> >
> > Details:
> >
> > Note that the following snippets are extremely trimmed down to 
> > eliminate irrelevant noise:
> >
> > vvvvv CONSUME ATSPI_KEY_PRESSED_EVENT: 'Up' (111) vvvvv
> > 16:27:57 - SPEECH OUTPUT: '16:14 link.'
> > Joanie: Caret nav puts you on the timestamp.
> > ^^^^^ CONSUME ATSPI_KEY_PRESSED_EVENT: 'Up' (111) ^^^^^
> >
> > vvvvv PROCESS OBJECT EVENT object:children-changed:add:system vvvvv
> > 16:27:57 - OBJECT EVENT: ... (2, 0, [tool bar | Message Actions])
> > Joanie: Messages Actions presumably popped up.
> > ^^^^^ PROCESS OBJECT EVENT object:children-changed:add:system ^^^^^
> >
> > vvvvv PROCESS OBJECT EVENT object:state-changed:focused vvvvv
> > 16:27:57 - OBJECT EVENT: ... (1, 0, 0) ... name='16:14' role='link'
> > 16:27:57 - WEB: Event ignored: Last command was caret nav
> > Joanie: Focus claim - set the caret on the link (already presented)
> > ^^^^^ PROCESS OBJECT EVENT object:state-changed:focused ^^^^^
> >
> > vvvvv CONSUME ATSPI_KEY_PRESSED_EVENT: 'Up' (111) vvvvv
> > 16:27:57 - WEB: Current context is: [link | 16:14], 0
> > 16:27:57 - ORCA: Changing locusOfFocus to [push button | ]
> > Joanie: Note this button lacks a name. That's not good. What is it?
> > 16:27:57 - SPEECH OUTPUT: 'push button.'
> > ^^^^^ CONSUME ATSPI_KEY_PRESSED_EVENT: 'Up' (111) ^^^^^
> >
> > vvvvv PROCESS OBJECT EVENT object:children-changed:add:system vvvvv
> > 16:27:57 - OBJECT EVENT: ... (2, 0, [tool bar | Message Actions])
> > Joanie: Messages Actions presumably popped up.
> > ^^^^^ PROCESS OBJECT EVENT object:children-changed:add:system ^^^^^
> >
> > vvvvv PROCESS OBJECT EVENT object:state-changed:focused vvvvv
> > 16:27:58 - OBJECT EVENT: ... (1, 0, 0) ... role='push button'
> > 16:27:58 - WEB: Event ignored: Last command was caret nav
> > Joanie: Focus claim - set the caret on the button (already presented)
> > ^^^^^ PROCESS OBJECT EVENT object:state-changed:focused ^^^^^
> >
> > vvvvv PROCESS OBJECT EVENT object:property-change:accessible-name vvvvv
> > 16:27:58 - ... Peter Vágner 16:13 I'm running Firefox nightly 74 and 
> > I haven't noticed it today. Message Actions)
> > Joanie: Orca does nothing in response. I wonder why this event came in.
> > ^^^^^ PROCESS OBJECT EVENT object:property-change:accessible-name ^^^^^
> >
> > vvvvv PROCESS OBJECT EVENT object:property-change:accessible-name vvvvv
> > 16:27:58 - ... 16:14 I was chatting for a hour or so with riot web in 
> > firefox 74.)
> > Joanie: Orca does nothing in response. I wonder why this event came in.
> > ^^^^^ PROCESS OBJECT EVENT object:property-change:accessible-name ^^^^^
> >
> > vvvvv CONSUME ATSPI_KEY_PRESSED_EVENT: 'Up' (111) vvvvv
> > 16:27:58 - WEB: Current context is: [push button | ], 0
> > 16:27:58 - ORCA: Changing locusOfFocus to [tool bar | Message Actions]
> > 16:27:58 - WEB: Generating speech contents (length: 2)
> > 16:27:58 - ITEM 0: [tool bar | ], start: 0, end: 1, string: ''
> > Joanie: Oops, where'd the name go?
> > 16:27:58 - ZOMBIE: [tool bar | ] is defunct
> > Joanie: Make that where did the toolbar go?
> > 16:27:58 - ITEM 1: [push button | ], start: 0, end: 1, string: ''
> > 16:27:58 - SPEECH OUTPUT: 'tool bar.'
> > 16:27:58 - SPEECH OUTPUT: 'push button.'
> > Joanie: This is where things have clearly gone south....
> > ^^^^^ CONSUME ATSPI_KEY_PRESSED_EVENT: 'Up' (111) ^^^^^
> >
> > Now we get events telling us the toolbar is dead:
> > * property-change:accessible-name for [tool bar | ]
> > * children-changed:remove:system  for [section } ] (2, 0, [tool bar | ])
> > * text-changed:delete:system for [section | ] (64, 1, [OBJ])
> >
> > And then the document itself claims focus because something's got to....
> > 16:27:58 - focus: for [document web | Riot [11]
> > 16:27:58 - state-changed:focused for [document web | Riot [11] (1, 0, 0)
> >
> > Orca soldiers on:
> >
> > vvvvv PROCESS OBJECT EVENT object:state-changed:focused vvvvv
> > 16:27:58 - OBJECT EVENT: ... (1, 0, 0) role='document web'
> > 16:27:58 - WEB: Caret context is [tool bar | ], 0 (focus: [tool bar | ])
> > 16:27:58 - ZOMBIE: [tool bar | ]'s index is -1
> > 16:27:58 - WEB: Clearing context - obj is null or zombie
> >
> > At this point Orca goes searching for where the caret should be.
> > It does this from the top down. Orca concludes the list (good).
> > But it *seems* to be settling on the first list item (not so good).
> > Not sure why yet.
> > ^^^^^ PROCESS OBJECT EVENT object:state-changed:focused ^^^^^
> >
> > At this point, when you up arrow, you wind up above the list. Which 
> > is what you reported, of course.
> >
> > So.... Maybe I can improve Orca's logic regarding trying to figure 
> > out what you're on -- or should be on -- when the thing you're on 
> > goes poof! Orca already does a lot of work in that regard, but it's 
> > clearly not working here....
> >
> > In the meantime, it would be really nice to be able to reproduce this 
> > myself, locally and reliably, so that I can debug it and try to 
> > improve that logic. So.... Would you, or the Riot developers, have 
> > any idea how I can make this toolbar gain focus and then get 
> > destroyed while one is on/in it?
> >
> > What would be even cooler is if the Riot developers can detect this 
> > condition themselves and maybe put focus or the caret where it 
> > belongs (i.e. in the current chat message associated with the toolbar 
> > which went poof!). Dunno if that's doable. But if it is, Orca should 
> > get an event from Firefox saying what has focus and/or the caret. And 
> > that would be a lot easier -- and way more performant in a super huge 
> > list like a chat log -- than trying to do an accessible tree dive for 
> > the active item.
> >
> > HTH.
> > --joanie
> >
> > On 1/22/20 5:34 PM, Nolan Darilek wrote:
> > > Very odd, you're not the only one unable to duplicate, seems to be 
> > > unique to me. I tried with both Firefox stable and nightly, 
> > > disabling all addons on nightly. Currently using riot.im/develop.
> > >
> > >
> > > Here's a debug.out of the behavior, with us talking about 
> > > duplicating it no less:
> > >
> > >
> > > https://send.firefox.com/download/91d0d5401798ef23/#MO0kXgEQYOG87-TngtlkcA 
> > >
> > >
> > >
> > > Thanks.
> > > On 1/22/20 4:24 PM, Joanmarie Diggs wrote:
> > > > Hey Nolan.
> > > >
> > > > I'm afraid I cannot reproduce the problem you describe. We use Riot 
> > > > at Igalia btw. I tried with Firefox Nightly and the following 
> > > > versions of Riot:
> > > >
> > > > matrix-react-sdk version: <local>
> > > > riot-web version: 1.5.6
> > > > olm version: 3.1.3
> > > >
> > > > Do you think something in particular might be triggering it, like 
> > > > an incoming message?
> > > >
> > > > --joanie
> > > >
> > > > On 1/22/20 10:25 AM, Nolan Darilek wrote:
> > > > > Working well here, thanks so much!
> > > > >
> > > > >
> > > > > There is another odd Riot issue that I don't think affects NVDA. 
> > > > > To duplicate:
> > > > >
> > > > >
> > > > > 1. Start with focus in the message entry area.
> > > > >
> > > > > 2. Switch to Browse mode and arrow up a few times (yay that I can 
> > > > > now do this! :)
> > > > >
> > > > > 3. After 4-5 up-arrows, focus flies out of the message list and 
> > > > > into the room header.
> > > > >
> > > > >
> > > > > I'd expect to be able to arrow up and read previous message 
> > > > > history. I can't always duplicate, but it happens way more often 
> > > > > than not. Using Orca master and Firefox 72.0.1.
> > > > >
> > > > >
> > > > > Thanks again.
> > > > >
> > > > > On 1/22/20 12:11 AM, Joanmarie Diggs wrote:
> > > > > > It looks like the Riot authors were creating a placeholder-like 
> > > > > > experience with actual elements which Orca was treating like real 
> > > > > > entry text and getting confused. This is now working for me. 
> > > > > > Please test and let me know.
> > > > > >
> > > > > > Thanks!
> > > > > > --joanie
> > > > > >
> > > > > > On 1/9/20 9:58 AM, Nolan Darilek wrote:
> > > > > > > Any chance this might get looked into again?
> > > > > > >
> > > > > > >
> > > > > > > FWIW, Mozilla is abandoning IRC for Matrix, so Riot is about to 
> > > > > > > become an indispensable tool for anyone interacting via chat in 
> > > > > > > the Mozilla ecosystem. They've also been actively working on 
> > > > > > > accessibility and have a good and improving NVDA story. I say 
> > > > > > > that not to pit Orca against NVDA, but rather to note that 
> > > > > > > they're really trying to fix their accessibility issues.
> > > > > > >
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 11/15/19 8:53 AM, Nolan Darilek wrote:
> > > > > > > > OK, was mistaken. The failure case is a bit different, but when 
> > > > > > > > I type into the field under Chromium, I can't arrow up past the 
> > > > > > > > previous message. This is under Chromium 78, which I was just 
> > > > > > > > upgraded to.
> > > > > > > >
> > > > > > > >
> > > > > > > > And yes yes, before you yell at me, I'm interested in this more 
> > > > > > > > as what now appears to be a cross-browser bug with slightly 
> > > > > > > > different behaviors, not as a Chromium issue. :) I just want 
> > > > > > > > this to work somewhere--I don't much care where.
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks.
> > > > > > > >
> > > > > > > >
> > > > > > > > On 11/13/19 5:52 PM, Nolan Darilek wrote:
> > > > > > > > > FWIW, I finally installed Chromium, and this doesn't happen 
> > > > > > > > > with Chromium 77.0.3865.120 Fedora Project and Orca master. So 
> > > > > > > > > seems specific to Firefox.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 11/5/19 3:05 PM, Joanmarie Diggs wrote:
> > > > > > > > > > I can reproduce this. It looks like an Orca bug. I'm still 
> > > > > > > > > > debugging it, but will hopefully have it fixed before too 
> > > > > > > > > > long. I don't think it's a regression though. Or if it is, 
> > > > > > > > > > it's super old. I went back to Orca 3.30 and the problem 
> > > > > > > > > > happens there....
> > > > > > > > > >
> > > > > > > > > > On 11/1/19 11:29 AM, Nolan Darilek wrote:
> > > > > > > > > > > Hey folks,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > The Riot developers have been pouring lots of effort 
> > > > > > > > > > > recently into making their web-based Matrix client 
> > > > > > > > > > > accessible. But I'm experiencing a pretty significant issue 
> > > > > > > > > > > with their new text entry system for messages that they 
> > > > > > > > > > > can't duplicate under NVDA. I'm wondering if any Orca users 
> > > > > > > > > > > are seeing the same, and if this can be diagnosed as either 
> > > > > > > > > > > a Firefox/Orca issue under Linux, or as something they 
> > > > > > > > > > > aren't doing correctly on their end. To test, you'll need a 
> > > > > > > > > > > Matrix account. Once you have one and are logged in, visit 
> > > > > > > > > > > this instance with accessibility improvements:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > https://riots.im/adhoc/a11y
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Join a room or chat. You'll probably be automatically 
> > > > > > > > > > > focused into the message input field and put in focus mode. 
> > > > > > > > > > > Once there, enter browse mode and try arrowing up.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Apparently, NVDA handles this use case fine both in the 
> > > > > > > > > > > accessibility build and the development build at 
> > > > > > > > > > > riot.im/develop. Orca, on the other hand, won't let me arrow 
> > > > > > > > > > > up. I hear "Newline," followed by some buttons, and focus 
> > > > > > > > > > > lands back in the message entry field. The only way to 
> > > > > > > > > > > escape this field is to enter browse mode and use b/B/h/some 
> > > > > > > > > > > other quick navigation key, then hope you don't land back in 
> > > > > > > > > > > the mud. Happens under Orca master as of a few minutes ago 
> > > > > > > > > > > and Firefox nightly as of a few days ago, but it's happened 
> > > > > > > > > > > for a bit now so I don't think it's a bleeding-edge 
> > > > > > > > > > > regression. Apologies for not reporting it--been a bit busy 
> > > > > > > > > > > and it hasn't been high enough priority for me.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Is there any actionable feedback I can give the Riot folks 
> > > > > > > > > > > to fix this? They're pretty actively looking for feedback in 
> > > > > > > > > > > the #a11y:matrix.org room, so if you're Matrix-inclined and 
> > > > > > > > > > > want to give it then there is the best place, or I'll 
> > > > > > > > > > > happily relay it otherwise.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Thanks a bunch.
> > > > > > > > > > >
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > orca-list mailing list
> > > > > > > > > > > orca-list gnome org
> > > > > > > > > > > https://mail.gnome.org/mailman/listinfo/orca-list
> > > > > > > > > > > Orca wiki: https://wiki.gnome.org/Projects/Orca
> > > > > > > > > > > Orca documentation: https://help.gnome.org/users/orca/stable/
> > > > > > > > > > > GNOME Universal Access guide: 
> > > > > > > > > > > https://help.gnome.org/users/gnome-help/stable/a11y.html
> > > > > > > _______________________________________________
> > > > > > > orca-list mailing list
> > > > > > > orca-list gnome org
> > > > > > > https://mail.gnome.org/mailman/listinfo/orca-list
> > > > > > > Orca wiki: https://wiki.gnome.org/Projects/Orca
> > > > > > > Orca documentation: https://help.gnome.org/users/orca/stable/
> > > > > > > GNOME Universal Access guide: 
> > > > > > > https://help.gnome.org/users/gnome-help/stable/a11y.html