Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UIA: selective event registration #11209

Closed
LeonarddeR opened this issue May 26, 2020 · 12 comments · Fixed by #11214
Closed

UIA: selective event registration #11209

LeonarddeR opened this issue May 26, 2020 · 12 comments · Fixed by #11214
Labels
api/uia Feature or bug requires an understanding of UIA component/event-handler performance
Milestone
2020.3

Comments

@LeonarddeR
Copy link
Collaborator

Related to #11077, #11109, #11002, #8742

Is your feature request related to a problem? Please describe.

NVDA is listening globally to many UIA elements. This causes a major performance hit when many events are received (#11109)

As part of #11077, I investigated a suggestion by @jcsteh to only queue events for objects of interest. However, this doesn't cause the major performance improvement I was hoping for.

Describe the solution you'd like

I'm going to prototype the following:

  1. An easier method for event registration by means of EventHandlerGroups. This feature was introduced in IUIAutomation6. I already wrote a FakeEventHandlerGroup class to mimic this behavior on older versions of UIA. In short, this should allow us to request for automation/ notification and property events in one call instead of three on newer versions of UIA.
  2. When changing focus, register for events fired at the focus object and its ancestors.
  3. distinguish between events that are of global interest (notifications, focus changes, value changes) and local interest (name, description, item status, etc.). We can have a global and a local event handler group. The global group will be registered once, the local group will be added/removed on focus.

Nothing is set in stone yet. I just want to tackle these issues as they have major impact on my daily work.
@LeonarddeR LeonarddeR added api/uia Feature or bug requires an understanding of UIA component/event-handler performance labels May 26, 2020
@josephsl
Copy link
Collaborator

josephsl commented May 26, 2020 via email

@zstanecic
Copy link
Contributor

zstanecic commented May 26, 2020 via email

@josephsl
Copy link
Collaborator

Hi,

The following Windows 10 versions are still being supported (as of time of this post):

  • Version 1507 (long-term only for mission-critical devices)
  • Version 1607/Server 2016 (long-term only for mission-critical devices and limited support for devices equipped with certain Intel Atom processors)
  • Version 1703 (Surface Hub only)
  • Version 1709 (extended support for businesses until October 2020)
  • Version 1803 (extended support for businesses until November 2020)
  • Version 1809/Server 2019 (extended support until November 2020 due to COVID-19, business support ends in May 2021)
  • Version 1903 (until December 2020)
  • Version 1909 (until May 2022)
  • Upcoming Version 2004 (prerelease, expected EOL is December 2021)

Thanks.

@LeonarddeR
Copy link
Collaborator Author

Here is a try build. note that I didn't test this properly on Windows 10 versions pre 1809.

I'm going to test this with #11109 now. I'm afraid it still doesn't fix #11002 though. For these, we need event limiting.

@codeofdusk
Copy link
Contributor

This does improve #11002 (at the very least, the console being flooded with events doesn't completely break UIA now)!

@josephsl
Copy link
Collaborator

josephsl commented May 27, 2020 •
edited
Loading

Hi,

This breaks modern input support, specifically no announcement from NVDA regarding the following:

  • Emoji panel closed
  • Moving among character categories in emoji panel
  • No dictation feedback

All of these rely on name change event coming from emoji panel header element, caused by the fact that NVDA will not see that emoji panel is the new foreground window. One workaround might be to let modern input app module (Composable Shell Input Panel in Version 1909, Text Input Host in Version 2004) register to listen to events coming from input features when the app module is instantiated. The thing is, unless killed, modern input support app will run in the background all the time, meaning that name change events will be received from all windows if not careful.

Thanks.

@LeonarddeR
Copy link
Collaborator Author

Is there an always available root element for the modern input stuff on which we can register another instance of the local event handler group,?

@josephsl
Copy link
Collaborator

josephsl commented May 27, 2020 via email •
edited
Loading

@lukaszgo1
Copy link
Contributor

Hi @LeonarddeR Thanks for your work. On Windows 7 x64 with this branch the error sound plays with each focus change and the following is written in the log:

ERROR - _UIAHandler.UIAHandler.MTAThreadFunc (11:42:03.421) - _UIAHandler.UIAHandler.MTAThread (1084):
Exception in function queued to UIA MTA thread
Traceback (most recent call last):
File "_UIAHandler.py", line 309, in MTAThreadFunc
func()
File "_UIAHandler.py", line 459, in
self.MTAThreadQueue.put_nowait(lambda: self._onFocusChange(previousFocusedUIAElement, sender))
File "_UIAHandler.py", line 375, in _onFocusChange
self.removeEventHandlerGroup(oldElement, self.localEventHandlerGroup)
File "_UIAHandler.py", line 366, in removeEventHandlerGroup
eventHandlerGroup.unregisterFromClientObject(element)
File "UIAUtils.py", line 277, in unregisterFromClientObject
self.clientObject.RemovePropertyChangedEventHandler(element, handler, properties)
NameError: name 'properties' is not defined

@beqabeqa473
Copy link
Contributor

beqabeqa473 commented May 27, 2020 via email

@LeonarddeR LeonarddeR mentioned this issue May 27, 2020
Merged
@LeonarddeR
Copy link
Collaborator Author

Here is a new try build that should fix issues with Windows 7 as reported by @lukaszgo1

@towebo
Copy link

towebo commented Jun 3, 2020

Thanks!
I used it at work today and I noticed a big difference. I'm running Visual Studio 2019 Community and working with WinForms applications. NVDA is much snappier in the errors list, in the assembly information dialog, in the solutions tree and so on.
The only back side I found was that NVDA didn't track text under mouse cursor in the Windows Command Prompt. Perhaps that's a different issue.

@nvaccessAuto nvaccessAuto added this to the 2020.3 milestone Jul 6, 2020
@codeofdusk codeofdusk mentioned this issue Feb 1, 2022
Merged
5 tasks
@codeofdusk codeofdusk mentioned this issue Aug 16, 2022
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
api/uia Feature or bug requires an understanding of UIA component/event-handler performance
Projects
None yet
Milestone
2020.3
Development

Successfully merging a pull request may close this issue.

Selective UIA event handling LeonarddeR/nvda
8 participants
@beqabeqa473 @codeofdusk @LeonarddeR @towebo @josephsl @lukaszgo1 @zstanecic @nvaccessAuto