Add a Fuzzing configuration and a version of conhost that can be fuzzed

[DHowett]

This commit introduces a new build configuration, "Fuzzing", which
enables the new address sanitizer (shipped in VS 16.9) and code
coverage over the entire solution. Only a small subset of projects
(those comprising original conhost, right now) are selected to build in
this configuration, and even then only in Fuzzing|x64.

It also adds a fuzzing-adapted build of conhost, which makes no server
connections and handles no client applications. To do this, I've
replicated a bit of the console startup routine into fuzzmain.cpp and
made up some fake data. This is the bare minimum required to boot up
Win32 interactivity (or VT interactivity!) and pretend that a process
has connected.

If we don't pretend that a process has connected, "conhost" will exit
immediately. If we don't forge the process list, conhost will exit. If
we can't provide a server handle, we can't provide a "device comm".

Minor changes were necessary to server/host such that they would accept
a preexisting "device comm". We use this new behavior to provide a
"null" one that only hangs up threads and otherwise responds to requests
successfully.

This fuzzing-adapted build links LLVM's libFuzzer, which is an excellent
coverage-based fuzzer that will produce a corpus of inputs that exercise
unique codepaths. Eventually, we can use this to generate known-"good"
inputs for anything.

I've gone ahead and added a fuzz function that yeets bytes directly into
WriteCharsLegacy, which was the original reason I went down this path.

The implementation of LLVMFuzzerTestOneInput should be replaced with
whatever you want to fuzz.

[github-actions]

Misspellings found, please review:

• asan
• fsanitize
• fuzzmain
• libsancov
• VAKUE

To accept these changes, run the following commands from this repository on this branch

pushd $(git rev-parse --show-toplevel)
perl -e '
my @expect_files=qw('".github/actions/spelling/expect/alphabet.txt
.github/actions/spelling/expect/expect.txt
.github/actions/spelling/expect/web.txt"');
@ARGV=@expect_files;
my @stale=qw('"aspnet boostorg BSODs Cac COINIT dahall fde fea fmtlib isocpp mintty NVDA pinam QOL remoting unte vcrt what3words xamarin "');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
  if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix"; rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv = $ARGV; }
  next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect/da24f7d9390a71648a81f17fc1eefc6e2d27bcbf.txt";
use File::Path qw(make_path);
make_path ".github/actions/spelling/expect";
open FILE, q{<}, $new_expect_file; chomp(my @words = <FILE>); close FILE;
my @add=qw('"asan cac coinit fsanitize fuzzmain libsancov Remoting VAKUE "');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a) cmp lc($b)} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE "$word\n" if $word =~ /\w/; };
close FILE;'
popd

✏️ Contributor please read this

By default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.

⚠️ The command is written for posix shells. You can copy the contents of each perl command excluding the outer ' marks and dropping any '"/"' quotation mark pairs into a file and then run perl file.pl from the root of the repository to run the code. Alternatively, you can manually insert the items...

If the listed items are:

• ... misspelled, then please correct them instead of using the command.
• ... names, please add them to .github/actions/spelling/dictionary/names.txt.
• ... APIs, you can add them to a file in .github/actions/spelling/dictionary/.
• ... just things you're using, please add them to an appropriate file in .github/actions/spelling/expect/.
• ... tokens you only need in one place and shouldn't generally be used, you can add an item in an appropriate file in .github/actions/spelling/patterns/.

See the README.md in each directory for more information.

🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉

🗜️ If you see a bunch of garbage and it relates to a binary-ish string, please add a file path to the .github/actions/spelling/excludes.txt file instead of just accepting the garbage.

File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

[DHowett]

these were the only changes necessary to conhost.
The only reason they are necessary is so that I can pass an invalid server handle 😄

[miniksa]

dev/duhowett/fuzzywuzzy

👀

[github-actions]

Misspellings found, please review:

• asan
• fsanitize
• fuzzmain
• libsancov
• VAKUE

To accept these changes, run the following commands from this repository on this branch

pushd $(git rev-parse --show-toplevel)
perl -e '
my @expect_files=qw('".github/actions/spelling/expect/alphabet.txt
.github/actions/spelling/expect/expect.txt
.github/actions/spelling/expect/web.txt"');
@ARGV=@expect_files;
my @stale=qw('"aspnet boostorg BSODs Cac COINIT dahall fde fea fmtlib isocpp mintty NVDA pinam QOL remoting unte vcrt what3words xamarin "');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
  if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix"; rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv = $ARGV; }
  next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect/da1e1a693e0358b7944e0c964d25c47fba0ce062.txt";
use File::Path qw(make_path);
make_path ".github/actions/spelling/expect";
open FILE, q{<}, $new_expect_file; chomp(my @words = <FILE>); close FILE;
my @add=qw('"asan cac coinit fsanitize fuzzmain libsancov Remoting VAKUE "');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a) cmp lc($b)} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE "$word\n" if $word =~ /\w/; };
close FILE;'
popd

✏️ Contributor please read this

By default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.

⚠️ The command is written for posix shells. You can copy the contents of each perl command excluding the outer ' marks and dropping any '"/"' quotation mark pairs into a file and then run perl file.pl from the root of the repository to run the code. Alternatively, you can manually insert the items...

If the listed items are:

• ... misspelled, then please correct them instead of using the command.
• ... names, please add them to .github/actions/spelling/dictionary/names.txt.
• ... APIs, you can add them to a file in .github/actions/spelling/dictionary/.
• ... just things you're using, please add them to an appropriate file in .github/actions/spelling/expect/.
• ... tokens you only need in one place and shouldn't generally be used, you can add an item in an appropriate file in .github/actions/spelling/patterns/.

See the README.md in each directory for more information.

🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉

🗜️ If you see a bunch of garbage and it relates to a binary-ish string, please add a file path to the .github/actions/spelling/excludes.txt file instead of just accepting the garbage.

File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

[WSLUser]

So is this the result of discussions mentioned in #7638 and will be part of CI?

[miniksa]

So is this the result of discussions mentioned in #7638 and will be part of CI?

Sort of. It just so happened to be Fix Hack Learn week this week and @DHowett was playing with fuzzers to try to tackle the WriteCharsLegacy problem while I attended a talk from the fuzzing team (who own OneFuzz) and am asking them how we can do some sort of CI that is GH-initiated instead of private-OS-side initiated. I have a meeting about it in the next week or two and I'd love to see us run some fuzz things in CI... though it might have to be nightly and not per-PR or rolling. We'll see.

[WSLUser]

though it might have to be nightly and not per-PR or rolling. We'll see.

I think nightly would be best to keep the CI from becoming too slow and each WT/conhost release should have the results published somewhere for the public to review. That GH Action they publish is still likely the best way forward.

[miniksa]

though it might have to be nightly and not per-PR or rolling. We'll see.

I think nightly would be best to keep the CI from becoming too slow and each WT/conhost release should have the results published somewhere for the public to review. That GH Action they publish is still likely the best way forward.

Thanks for your input.

[DHowett]

I've marked this PR as "ready for review" and filled out the body.

[miniksa]

Looks pretty good. Just the two comments but I trust you to resolve them before hitting merge so I'm just gonna leave the Approve check.

[zadjii-msft]

I look forward to messing up the configuration on more projects in the future

[zadjii-msft]

this one's got a weird mix of debug/fuzzing

[zadjii-msft]

Same with this one, but it's all Any Cpu so maybe that doesn't matter?

[zadjii-msft]

another mixed|any cpu

[zadjii-msft]

curiously mixed (release, fuzzing)|any cpu

[DHowett]

So @zadjii-msft, the mixed weird tuples are because those projects are just not selected to build in the fuzzing config. The burden of fixing the mapping would be on the person who enables one of them :)

anyCPU is the most annoying obviously, since it’s a CPU type that only 3 of our 3,100 projects have so we need to carry it around like an empty husk

[zadjii-msft]

we should almost just add a separate sln for those 😐

[DHowett]

Validation Steps Performed

I've been running conhost fuzzing for hours!

Right now, the WCL fuzzer is completely stateful since we never clear/reset the buffer. It appends and appends and appends and leaks heaps of memory (for some reason...).

The code in the fuzz function is written in a very "if you can find it, use it" way. I'm calling weird stuff from all over the codebase because it does what we need.

I have been treating this as a magic input generator. If I am looking at WriteCharsLegacy, and I wonder "what sort of input will get me to this line of code?" I can literally set a breakpoint and half a second later I'm staring at inputs that got me there. It's freaking amazing.

[ghost]

Hello @DHowett!

Because this pull request has the AutoMerge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.