Bug Report: What is reordering VT escape sequences in my string?

[jazzdelightsme]

Environment

Windows build number: Microsoft Windows [Version 10.0.18362.239]
Windows Terminal version (if applicable): (built from latest source as of July 17th, 2019, morning)

I am working on implementing some new VT escape sequences (PR here), but I've run into a strange problem. What the sequences do (or are supposed to do) is not important here. The problem is that in certain circumstances, something is reordering VT escape sequences in my content (which is very surprising to me).

The control sequences in question are XTPUSHSGR and XTPOPSGR: [CSI]#p and [CSI]#q, respectively. Note in the string below, how there is an XTPUSHSGR sequence, then a normal SGR sequence to set the color to red, then an XTPOPSGR sequence (and then a normal SGR sequence to reset the color).

When I open OpenConsole.sln in Visual Studio 2019, with the CascadiaPackage set as the default project to debug, and then F5 to start debugging it, it launches a terminal running Windows PowerShell. When I send my string to the console, something is moving the XTPUSHSGR and XTPOPSGR sequences to the very beginning of the string!

Steps to reproduce

1. Start OpenConsole.sln in Visual Studio 2019. Make sure CascadiaPackage is the default project, and platform is set to x64, and build the solution if necessary.
2. F5 to start debugging a new instance. You should now have a PowerShell instance in the terminal.
3. In the PowerShell instance in the terminal, run the following commands:

 $esc = [char] 0x1b
 $s = "Hello. Push:${esc}[#p ${esc}[91m this text is RED Pop:${esc}[#q FG should " +
      "no longer be red (unless it was red before) courtesy reset: ${esc}[0m"

4. Set a breakpoint at TerminalControl.dll!Microsoft::Console::VirtualTerminal::StateMachine::ProcessString (the two-parameter one).
5. In the PowerShell instance in the terminal, run the command $s to output the string.
6. When you hit the breakpoint, observe the value of rgwch.

Expected behavior

The content of the string should be just as it was set in PowerShell.

Actual behavior

The XTPUSHSGR and XTPOPSGR sequences have been moved to the beginning of the string!

My first thought was that PowerShell must be doing something funny, but I attached windbg to the PowerShell and set a breakpoint on kernel32!WriteConsoleW, and the string passed in there is correct. So something is happening after the string has left PowerShell's hands.

[zadjii-msft]

I'm willing to bet serious money that when conhost in conpty mode processes a long string with VT sequences in it that conhost doesn't understand, we render the failing VT sequences before we repaint for the rest of the string.

So for your example string: "Hello. Push:${esc}[#p ${esc}[91m this text is RED Pop:${esc}[#q FG should no longer be red (unless it was red before) courtesy reset: ${esc}[0m"

Conhost is going to try and process that entire string in one frame. Conpty is only going to render the output at the end of a frame, once every 60th of a second. However, as soon as the adapter (adaptDispatch) sees a sequence it doesn't understand, it will dispatch it to the terminal immediately. So the push/pops will appear in the stream before the rest of the frame is rendered.

Possible solutions:

• A workaround might just be to split the push/pops into separate strings to write to the terminal. It might trigger a frame in between them and the actual text, and it'll appear correctly.

$push = "${esc}[#p"
$red = "${esc}[91m"
$pop = "${esc}[#q"
$reset = "${esc}[0m"
$s1 = "Hello. Push:"
$s2 = "this text is RED Pop:${esc}[#q FG should no longer be red (unless it was red before) courtesy reset: ${esc}[0m"
$s3 = "FG should no longer be red (unless it was red before) courtesy reset: "

• A more correct solution would be to have conpty actually be able to process those sequences, though this issue will remain for any sequences conhost doesn't understand
• Maybe the most correct solution would be to have conpty force a frame when it encounters a sequence it doesn't recognize. That might really hit perf though, so I'd be cautious with that change.

[jazzdelightsme]

Whoa. Interesting. Thanks for the explanation, @zadjii-msft ; I would never have figured that out.

Part of my problem seems to be a debugging problem: in my branch with some code changes to handle those sequences, I implemented the push/pop commands in the Terminal class, but in the adaptDispatch stuff, I just returned false. I had set breakpoints in the adaptDispatch functions... but when debugging the CascadiaPackage, those breakpoints are disabled, claiming that no symbols have been loaded for the document, so I thought that code was not even running.

But the code definitely IS running... if I change the adaptDispatch layer to just return true (without actually doing anything), the behavior changes!

Although I don't understand the changed behavior... now the push/pop control sequences are simply gone. Should the adaptDispatch layer be doing something to send the sequences on down the line?

[zadjii-msft]

Oh that's because debugging the Terminal's conhost is a pain. Basically, Windows Terminal is running attached to another conhost process, and that conhost process is where adaptDispatch is running. To be able to debug it with VS, you need to manually attach VS to the child conhost process that the Terminal started.

[DHowett-MSFT]

Marking this one triaged; it's definitely something we should look at, but we may not have the bandwidth to do so.