Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attach UiaRenderer and Fire Selection Changed Events #2989

Merged
merged 21 commits into from
Dec 11, 2019

Conversation

carlos-zamora
Copy link
Member

@carlos-zamora carlos-zamora commented Sep 30, 2019

Summary of the Pull Request

I recommend looking at this PR by commit.

This PR makes use of the UiaRenderer by attaching it to the TerminalControl and setting up selectionChanged events for accessibility.

References

#2447

PR Checklist

  • Closes none
  • CLA signed.
  • Tests added/passed
  • Requires documentation to be updated

Detailed Description of the Pull Request / Additional comments

Commit 1: attaching the UiaRenderer

The uiaRenderer is treated very similarly to the dxRenderer. We have a unique_ptr ref to it in the TermControl. This gets populated when the TermControlAutomationPeer is created (thus enabling accessibility).

To prevent every TermControl from sending signals simultaneously, we specifically only enable whichever one is in an active pane.

The UiaRenderer needs to send encoded events to the automation provider (in this case, TermControlAutomationPeer). We needed our own automation events so that we can reuse this model for ConHost. This is the purpose of IUiaEventDispatcher.

We need a dispatcher for the UiaRenderer. Otherwise, we would do a lot of work to find out when to fire an event, but we wouldn't have a way of doing that.

Commit 2: hooking up selection events

This provides a little bit of polish to hooking it up before. Primarily to actually make it work. This includes returning S_FALSE instead of E_NOTIMPL.

The main thing here really is just how to detect if a selection has changed. This also shows how clean adding more events will be in the future!

Validation Steps Performed

  1. Launch Windows Terminal
  2. Launch Narrator (WIN+CTRL+Enter)

Scenario 1: cell selection

click on a cell with text in it
Narrator should read it out

Scenario 2a: expand selection

Click on a cell with text in it and drag it over more text (extending the selection)
Narrator shouild read the first cell, then read either...

  • the entire selection
  • what was appended to the selection
    (this is still unclear to me when it does which one)

Scenario 2b: reduce selection

Make a selection of an entire word (i.e.: "Powershell")
Reduce the selection to the first character
Narrator should read what got unselected

Scenario 3: Double click

double click on a word
Narrator should read the entire selection

Scenario 4: Triple click

Triple click on a line
Narrator should read the entire selection

@carlos-zamora carlos-zamora added the Area-Accessibility Issues related to accessibility label Sep 30, 2019
@carlos-zamora carlos-zamora requested a review from a team September 30, 2019 16:40
@carlos-zamora carlos-zamora self-assigned this Sep 30, 2019
@carlos-zamora carlos-zamora changed the title Dev/cazamor/acc/sig/select Attach UiaRenderer and Fire Selection Changed Events Sep 30, 2019
@DHowett-MSFT
Copy link
Contributor

/azp run

@azure-pipelines
Copy link

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

Copy link
Member

@zadjii-msft zadjii-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have lots of comments/questions, but considering I'll be gone in 3 days, I don't want to block the PR

src/types/IUiaEventDispatcher.h Outdated Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Outdated Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Outdated Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Outdated Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Outdated Show resolved Hide resolved
src/cascadia/TerminalControl/TermControl.cpp Show resolved Hide resolved
src/cascadia/TerminalControl/TermControl.cpp Show resolved Hide resolved
src/cascadia/TerminalControl/TermControl.cpp Outdated Show resolved Hide resolved
src/cascadia/TerminalControl/TermControl.cpp Show resolved Hide resolved
src/cascadia/TerminalControl/TermControl.cpp Outdated Show resolved Hide resolved
src/cascadia/TerminalControl/TermControl.cpp Show resolved Hide resolved
src/types/IUiaEventDispatcher.h Outdated Show resolved Hide resolved
@carlos-zamora carlos-zamora mentioned this pull request Oct 15, 2019
4 tasks
@carlos-zamora
Copy link
Member Author

Whoops. That last commit was intended for the other branch... but eh. It's all going to the same place anyways.

@carlos-zamora carlos-zamora changed the base branch from dev/cazamor/acc/signaling to master October 28, 2019 16:44
@carlos-zamora
Copy link
Member Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Member

@zadjii-msft zadjii-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm marking as changes requested to make sure the _prevSelection thing gets followed up on. I'm open to being overruled on the SignalUia thing.

src/renderer/uia/lib/uia.vcxproj Outdated Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Outdated Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Show resolved Hide resolved
src/types/IUiaEventDispatcher.h Outdated Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Show resolved Hide resolved
@ghost ghost added Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something and removed Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something labels Dec 5, 2019
Copy link
Member

@zadjii-msft zadjii-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of this is worth blocking over, but I don't know if these are serious concerns or not.

src/renderer/uia/UiaRenderer.cpp Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Show resolved Hide resolved
src/renderer/uia/UiaRenderer.cpp Show resolved Hide resolved
src/types/IUiaEventDispatcher.h Show resolved Hide resolved
Copy link
Member

@miniksa miniksa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's still a few things outstanding, so I don't want to sign off yet.

@carlos-zamora carlos-zamora merged commit 2b8b034 into master Dec 11, 2019
@carlos-zamora carlos-zamora deleted the dev/cazamor/acc/sig/select branch December 11, 2019 21:52
@ghost
Copy link

ghost commented Jan 14, 2020

🎉Windows Terminal Preview v0.8.10091.0 has been released which incorporates this pull request.:tada:

Handy links:

@carlos-zamora carlos-zamora mentioned this pull request Feb 3, 2020
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Accessibility Issues related to accessibility
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants