SARIF_LEVEL_OVERRIDE, SARIF_KIND_OVERRIDE

[michaelcfanning]

When we complete #2228, we should add support for environment variables that completely replace anything on the command-line if they are encountered. So,

set SARIF_LEVEL_OVERRIDE=Error;Warning;Note

could be used to automatically opt into 'experimental' failure messages (which has the 'note' level set on them and would otherwise be filtered). this is useful to look at the behavior of existing analysis systems without requiring that tools be reconfigured.

Notes:

• We should think about a name convention that maps to arg names. I've chosed SARIF_[ARGNAME]_OVERRIDE. We could use this same standard for any/all arguments we decide can be overridden.
• It is pretty dangerous to conditionally drive tool behavior based on environment variables. One protection against this is to require some explicit command-line argument that allows for these variables to be read/override arguments. That requires users to update all their command-lines to enable the functionality before it works.
• Requiring opt-in to this override policy could be burdensome. If we do not require it, we should definitely emit an extremely prominent notification which can't be disabled via all our magical mechanisms that reports that an argument is being overridden. Error notifications in particular can never be disabled. If we raise an error notification for this, however, the experiment might break systems that halt on errors, preventing users from experimenting with lighting up non-default analysis. So as usual, there's more nuance here than one might think.