Skip to content

Commit

Permalink
Attach UiaRenderer and Fire Selection Changed Events (#2989)
Browse files Browse the repository at this point in the history 
This PR makes use of the UiaRenderer by attaching it to the TerminalControl and setting up selectionChanged events for accessibility.

Part 1: attaching the UiaRenderer
The uiaRenderer is treated very similarly to the dxRenderer. We have a unique_ptr ref to it in the TermControl. This gets populated when the TermControlAutomationPeer is created (thus enabling accessibility).

To prevent every TermControl from sending signals simultaneously, we specifically only enable whichever one is in an active pane.

The UiaRenderer needs to send encoded events to the automation provider (in this case, TermControlAutomationPeer). We needed our own automation events so that we can reuse this model for ConHost. This is the purpose of IUiaEventDispatcher.

We need a dispatcher for the UiaRenderer. Otherwise, we would do a lot of work to find out when to fire an event, but we wouldn't have a way of doing that.

Part 2: hooking up selection events
This provides a little bit of polish to hooking it up before. Primarily to actually make it work. This includes returning S_FALSE instead of E_NOTIMPL.

The main thing here really is just how to detect if a selection has changed. This also shows how clean adding more events will be in the future!
  • Loading branch information
@carlos-zamora
carlos-zamora authored Dec 11, 2019
1 parent cb17dae commit 2b8b034
Show file tree
Hide file tree
Showing 11 changed files with 204 additions and 56 deletions.
54 changes: 38 additions & 16 deletions src/cascadia/TerminalControl/TermControl.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
using namespace ::Microsoft::Console::Types;
using namespace ::Microsoft::Terminal::Core;
using namespace winrt::Windows::UI::Xaml;
using namespace winrt::Windows::UI::Xaml::Automation::Peers;
using namespace winrt::Windows::UI::Core;
using namespace winrt::Windows::System;
using namespace winrt::Microsoft::Terminal::Settings;
Expand Down Expand Up @@ -393,20 +394,31 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
Close();
}

// Method Description:
// - Creates an automation peer for the Terminal Control, enabling accessibility on our control.
// Arguments:
// - None
// Return Value:
// - The automation peer for our control
Windows::UI::Xaml::Automation::Peers::AutomationPeer TermControl::OnCreateAutomationPeer()
try
{
Windows::UI::Xaml::Automation::Peers::AutomationPeer autoPeer{ nullptr };
if (GetUiaData())
{
try
{
// create a custom automation peer with this code pattern:
// (https://docs.microsoft.com/en-us/windows/uwp/design/accessibility/custom-automation-peers)
autoPeer = winrt::make<winrt::Microsoft::Terminal::TerminalControl::implementation::TermControlAutomationPeer>(this);
}
CATCH_LOG();
// create a custom automation peer with this code pattern:
// (https://docs.microsoft.com/en-us/windows/uwp/design/accessibility/custom-automation-peers)
auto autoPeer = winrt::make_self<winrt::Microsoft::Terminal::TerminalControl::implementation::TermControlAutomationPeer>(this);

_uiaEngine = std::make_unique<::Microsoft::Console::Render::UiaEngine>(autoPeer.get());
_renderer->AddRenderEngine(_uiaEngine.get());
return *autoPeer;
}
return autoPeer;
return nullptr;
}
catch (...)
{
LOG_CAUGHT_EXCEPTION();
return nullptr;
}

::Microsoft::Console::Types::IUiaData* TermControl::GetUiaData() const
Expand Down Expand Up @@ -780,22 +792,20 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
if (multiClickMapper == 3)
{
_terminal->TripleClickSelection(terminalPosition);
_renderer->TriggerSelection();
}
else if (multiClickMapper == 2)
{
_terminal->DoubleClickSelection(terminalPosition);
_renderer->TriggerSelection();
}
else
{
// save location before rendering
_terminal->SetSelectionAnchor(terminalPosition);

_renderer->TriggerSelection();
_lastMouseClick = point.Timestamp();
_lastMouseClickPos = cursorPosition;
}
_renderer->TriggerSelection();
}
else if (point.Properties().IsRightButtonPressed())
{
Expand Down Expand Up @@ -1195,8 +1205,9 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
}

// Method Description:
// - Event handler for the GotFocus event. This is used to start
// blinking the cursor when the window is focused.
// - Event handler for the GotFocus event. This is used to...
// - enable accessibility notifications for this TermControl
// - start blinking the cursor when the window is focused
void TermControl::_GotFocusHandler(Windows::Foundation::IInspectable const& /* sender */,
RoutedEventArgs const& /* args */)
{
Expand All @@ -1206,6 +1217,11 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
}
_focused = true;

if (_uiaEngine.get())
{
THROW_IF_FAILED(_uiaEngine->Enable());
}

if (_tsfInputControl != nullptr)
{
_tsfInputControl.NotifyFocusEnter();
Expand All @@ -1218,8 +1234,9 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
}

// Method Description:
// - Event handler for the LostFocus event. This is used to hide
// and stop blinking the cursor when the window loses focus.
// - Event handler for the LostFocus event. This is used to...
// - disable accessibility notifications for this TermControl
// - hide and stop blinking the cursor when the window loses focus.
void TermControl::_LostFocusHandler(Windows::Foundation::IInspectable const& /* sender */,
RoutedEventArgs const& /* args */)
{
Expand All @@ -1229,6 +1246,11 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
}
_focused = false;

if (_uiaEngine.get())
{
THROW_IF_FAILED(_uiaEngine->Disable());
}

if (_tsfInputControl != nullptr)
{
_tsfInputControl.NotifyFocusLeave();
Expand Down
2 changes: 2 additions & 0 deletions src/cascadia/TerminalControl/TermControl.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#include <winrt/Microsoft.Terminal.Settings.h>
#include "../../renderer/base/Renderer.hpp"
#include "../../renderer/dx/DxRenderer.hpp"
#include "../../renderer/uia/UiaRenderer.hpp"
#include "../../cascadia/TerminalCore/Terminal.hpp"
#include "cppwinrt_utils.h"

Expand Down Expand Up @@ -113,6 +114,7 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation

std::unique_ptr<::Microsoft::Console::Render::Renderer> _renderer;
std::unique_ptr<::Microsoft::Console::Render::DxEngine> _renderEngine;
std::unique_ptr<::Microsoft::Console::Render::UiaEngine> _uiaEngine;

Settings::IControlSettings _settings;
bool _focused;
Expand Down
42 changes: 42 additions & 0 deletions src/cascadia/TerminalControl/TermControlAutomationPeer.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,48 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
THROW_IF_FAILED(::Microsoft::WRL::MakeAndInitialize<::Microsoft::Terminal::TermControlUiaProvider>(&_uiaProvider, owner, std::bind(&TermControlAutomationPeer::GetBoundingRectWrapped, this)));
};

// Method Description:
// - Signals the ui automation client that the terminal's selection has changed and should be updated
// Arguments:
// - <none>
// Return Value:
// - <none>
void TermControlAutomationPeer::SignalSelectionChanged()
{
Dispatcher().RunAsync(Windows::UI::Core::CoreDispatcherPriority::Normal, [&]() {
// The event that is raised when the text selection is modified.
RaiseAutomationEvent(AutomationEvents::TextPatternOnTextSelectionChanged);
});
}

// Method Description:
// - Signals the ui automation client that the terminal's output has changed and should be updated
// Arguments:
// - <none>
// Return Value:
// - <none>
void TermControlAutomationPeer::SignalTextChanged()
{
Dispatcher().RunAsync(Windows::UI::Core::CoreDispatcherPriority::Normal, [&]() {
// The event that is raised when textual content is modified.
RaiseAutomationEvent(AutomationEvents::TextPatternOnTextChanged);
});
}

// Method Description:
// - Signals the ui automation client that the cursor's state has changed and should be updated
// Arguments:
// - <none>
// Return Value:
// - <none>
void TermControlAutomationPeer::SignalCursorChanged()
{
Dispatcher().RunAsync(Windows::UI::Core::CoreDispatcherPriority::Normal, [&]() {
// The event that is raised when the text was changed in an edit control.
RaiseAutomationEvent(AutomationEvents::TextEditTextChanged);
});
}

winrt::hstring TermControlAutomationPeer::GetClassNameCore() const
{
return L"TermControl";
Expand Down
12 changes: 9 additions & 3 deletions src/cascadia/TerminalControl/TermControlAutomationPeer.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,14 @@ Author(s):
#include "TermControl.h"
#include "TermControlAutomationPeer.g.h"
#include <winrt/Microsoft.Terminal.TerminalControl.h>
#include "../../renderer/inc/IRenderData.hpp"
#include "../types/WindowUiaProviderBase.hpp"
#include "TermControlUiaProvider.hpp"
#include "../types/IUiaEventDispatcher.h"

namespace winrt::Microsoft::Terminal::TerminalControl::implementation
{
struct TermControlAutomationPeer :
public TermControlAutomationPeerT<TermControlAutomationPeer>
public TermControlAutomationPeerT<TermControlAutomationPeer>,
::Microsoft::Console::Types::IUiaEventDispatcher
{
public:
TermControlAutomationPeer(winrt::Microsoft::Terminal::TerminalControl::implementation::TermControl* owner);
Expand All @@ -44,6 +44,12 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
winrt::hstring GetLocalizedControlTypeCore() const;
winrt::Windows::Foundation::IInspectable GetPatternCore(winrt::Windows::UI::Xaml::Automation::Peers::PatternInterface patternInterface) const;

#pragma region IUiaEventDispatcher
void SignalSelectionChanged() override;
void SignalTextChanged() override;
void SignalCursorChanged() override;
#pragma endregion

#pragma region ITextProvider Pattern
Windows::UI::Xaml::Automation::Provider::ITextRangeProvider RangeFromPoint(Windows::Foundation::Point screenLocation);
Windows::UI::Xaml::Automation::Provider::ITextRangeProvider RangeFromChild(Windows::UI::Xaml::Automation::Provider::IRawElementProviderSimple childElement);
Expand Down
7 changes: 3 additions & 4 deletions src/cascadia/TerminalControl/TermControlAutomationPeer.idl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,9 @@ import "TermControl.idl";

namespace Microsoft.Terminal.TerminalControl
{
[default_interface]
runtimeclass TermControlAutomationPeer :
[default_interface] runtimeclass TermControlAutomationPeer :
Windows.UI.Xaml.Automation.Peers.FrameworkElementAutomationPeer,
Windows.UI.Xaml.Automation.Provider.ITextProvider
{
}
{
}
}
1 change: 1 addition & 0 deletions src/cascadia/TerminalControl/TerminalControl.vcxproj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
<ProjectReference Include="..\..\buffer\out\lib\bufferout.vcxproj" />
<ProjectReference Include="$(OpenConsoleDir)src\renderer\base\lib\base.vcxproj" />
<ProjectReference Include="..\..\renderer\dx\lib\dx.vcxproj" />
<ProjectReference Include="..\..\renderer\uia\lib\uia.vcxproj" />
<ProjectReference Include="..\..\terminal\parser\lib\parser.vcxproj" />
<ProjectReference Include="$(OpenConsoleDir)src\terminal\input\lib\terminalinput.vcxproj" />
<ProjectReference Include="$(OpenConsoleDir)src\cascadia\TerminalSettings\TerminalSettings.vcxproj">
Expand Down
Loading

1 comment on commit 2b8b034

@Rrogntudju
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this merge, the package would not build on my PC with a linker error : ConRenderUia.lib was missing. To make it build, i had to modify the build dependencies of the TerminalControl project, selecting RendererUia

Please sign in to comment.