Inefficient codegen for certain op and comparison

[tannergooding]

As per dotnet/coreclr#20788 (comment), we are currently generating some inefficient code for certain op and comparison scenarios.

For example, on if ((xbits ^ ybits) < 0), we are currently generating:

xor      edx, eax
test     edx, edx
jge      SHORT G_M7953_IG04

When we could get rid of the test and just generate:

xor edx, eax
jns SHORT G_M7953_IG04

category:cq
theme:codegen
skill-level:intermediate
cost:small

[anthonycanino]

Hi @tannergooding , I've been looking into this issue over the past few days and it looks like a lot of work has been done for this optimization since this issue was posted.

It looks like dotnet/coreclr#14027 started some logic for reusing flags during the lowering stage, at which point some of the cases had to be restricted due to the OF #9059. There was some discussion about making this work for ADD/SUB + LT/GE but it looks like it was abandoned due to low optimization opportunity #6794 (comment).

More recently, a PR #38586 added flag tracking during the codegen/emit stage, and addressed some remaining optimization cases, e.g., (x ^ y) < 0.

I think with PR 38586, we can handle cases like if (x - y) < 0, or more importantly, for (int x = v; x >= 0; x -= 2) as we can now look at whether the previous instruction set the SF, and if we are in a GT_GE/GT_LT with a SLT,/SGE and instead emit a JS/JNS instead.

Any tips on whether this optimization is worth pursing, as it is 1 or 2 of the remaining cases left (from my understanding).

[anthonycanino]

@tannergooding can we close this issue now per the PR merge?