UIA freezes after a large number of events is received

[codeofdusk]

Steps to reproduce:

1. Enable "use UI Automation to access the Windows Console" from advanced preferences.

2. Run cmd.

3. Start a python interpreter: python.

4. Run:

   for i in range(100000):
   print("test")

Actual behavior:

"test" is read for a few seconds, then all UIA applications become completely inaccessible until NVDA is restarted.

Expected behavior:

Output stops for a few seconds, then UIA recovers.

Context

I also tried disabling UIA connection recovery and event coalescing with the following in the NVDA Python console:

import UIAHandler
UIAHandler.handler.clientObject.CoalesceEvents = 0
UIAHandler.handler.clientObject.ConnectionRecoveryBehavior = 0

After UIA dies, running uiaHandler.terminate() and UIAHandler.initialize() does not restart UIA functionality. I'm not sure how to detect when UIA breaks or how to restart it once it does.

System configuration

NVDA installed/portable/running from source:

Installed and source.

NVDA version:

Latest master (b38776a).

Windows version:

Windows 10 21H1 insider.

Name and version of other software in use when reproducing the issue:

Windows Console

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Yes.

Have you tried any other versions of NVDA? If so, please report their behaviors.

2019.2.1, 2019.3, 2020.1 (all same result).

If addons are disabled, is your problem still occuring?

Yes.

Did you try to run the COM registry fixing tool in NVDA menu / tools?

Not applicable.

[codeofdusk]

Narrator seems to work as expected here, at least with my test case.

Cc @josephsl @LeonarddeR @michaelDCurran.

[LeonarddeR]

Hmm, this might be related to other UIA performance issues. cc @bramd

[francipvb]

Users of Visual Studio are experiencing performance issues with UIAutomation, too... And with Win10 native UI in general.

…

-- Cuando tus fuerzas terminan, las de mi Dios comienzan.

[LeonarddeR]

I have the strong feeling that something @jcsteh wrote in #10508 (comment) is related to this:

Obviously, not listening to property change events altogether isn't an option. However, Narrator and JAWS don't seem to be impacted like NVDA is. Some thoughts on that:

1. This is another reason to again consider selectively registering for property change events (e.g. on focus + ancestors and progress bar value changes), rather than unconditionally listening to all of them.

2. If I Register for property change events but return at the top of IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (so we receive the event but discard it), it's pretty responsive. That suggests we might be able to optimise some of the stuff we do in the event handler; e.g. constructing the object, choosing overlay classes, etc.

I would say that a solution based on option 2 would have the least chance of regressions while having positive impact on performance.

Now, I"m not sure what thread or threads handle all these events, as the log calls them dummy threads. However, if the UIA handler is processing an event and might be unable to process the next, if processing one event takes too much time without us discarding events that are out of date, things are slowing down. I think this is exactly what we're experiencing in several situations now, particularly in Visual Studio and may be in Word as well.

Cc @michaelDCurran @josephsl @bramd

[beqabeqa473]

yes, in most of the modern uia applications, nvda has serious performance issues. visual studio is a good example of this. also skype is also infected, when openning new windows, nvda is silent for several seconds. jaws doesn't have such problems.

…

On 4/21/20, Leonard de Ruijter ***@***.***> wrote: I have the strong feeling that something @jcsteh wrote in #10508 (comment) is related to this: > Obviously, not listening to property change events altogether isn't an > option. However, Narrator and JAWS don't seem to be impacted like NVDA is. > Some thoughts on that: > > 1. This is another reason to again consider selectively registering > for property change events (e.g. on focus + ancestors and progress bar > value changes), rather than unconditionally listening to all of them. > > 2. If I Register for property change events but return at the top of > IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (so we > receive the event but discard it), it's pretty responsive. That suggests > we might be able to optimise some of the stuff we do in the event handler; > e.g. constructing the object, choosing overlay classes, etc. I would say that a solution based on option 2 would have the least chance of regressions while having positive impact on performance. Now, I"m not sure what thread or threads handle all these events, as the log calls them dummy threads. However, if the UIA handler is processing an event and might be unable to process the next, if processing one event takes too much time without us discarding events that are out of date, things are slowing down. I think this is exactly what we're experiencing in several situations now, particularly in Visual Studio and may be in Word as well. Cc @michaelDCurran @josephsl @bramd -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #11002 (comment)

-- with best regards beqa

[LeonarddeR]

Related to #11077

[codeofdusk]

At least for consoles, a lot of this appears to be a result of us registering for the textChange and caret events in particular: rejecting them in EventHandler.shouldAcceptEvent does not stop NVDA's UIA support from hanging, but commenting out their entries in UIAHandler.UIAEventIdsToNVDAEventNames allows consoles with lots of text not to flood the system at the cost of autoread support.

Since consoles won't need these events any more once UIA notifications support is implemented for functional autoread, we should consider a way for NVDA not to register for these events from certain spammy providers, or only register for them in known-good cases. Thoughts @LeonarddeR, @michaelDCurran, @carlos-zamora?

[seanbudd]

Potentially related to #13334

[codeofdusk]

Potentially related to #13334

Not quite, because Notepad on Windows 10 uses MSAA, not UIA. Notepad on Windows 11 is UIA based, and if this hang can be reproduced there it is caused by this issue. Similar issues in VS, Windows Terminal, and (UIA) console are also in scope.

[codeofdusk]

Further investigation is required of course, but I think something along the lines of #10556 for UIA events would go a very long way here.

[Adriani90]

This has been probably fixed by #14888. Closing as fixed, but feel free to comment in case the issue still exists in the last NVDA alpha version and we can reopen.

@LeonarddeR now that #14888 is merged, is selective registering for UIA events still needed at all? You implemented that in #11214.

[LeonarddeR]

@LeonarddeR now that #14888 is merged, is selective registering for UIA events still needed at all? You implemented that in #11214.

Yes, I think so. It generally doesn't make sense to register to events you don't need.