Dynamic PGO work planned for .NET 7

[AndyAyersMS]

This issue captures the planned work for .NET 7. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.

Summary

Dynamic PGO is an opt-in feature introduced in .NET 6. In .NET 7 we will work on making Dynamic PGO easier to use, applicable to more scenarios, and offering more performance improvements.

As part of this work, we will also enable On-Stack Replacement (OSR) and change the default for QuickJitForLoops for x64 and arm64. This will improve Dynamic PGO by including methods with loops in the Tier0 set. It will also notably improve startup time for apps that have jit-intensive startup interval.

Planned for .NET 7

• (EgorBo WIP) Add support for enabling Dynamic PGO in SDK projects and runtime configs (Proposal: Public PGO config switch #70410, Introduce System.Runtime.TieredPGO knob #71438, Introduce System.Runtime.TieredPGO knob sdk#26350)
• (Aman WIP) Hot/Cold splitting (Implement fake hot/cold splitting and corresponding stress mode #69763, Add hot/cold splitting test job to jit-runtime-experimental #69922, Enable fake hot/cold splitting on ARM64 #70708, Enable hot/cold splitting of EH funclets #71236, Disable HANDLER_ENTRY_MUST_BE_IN_HOT_SECTION #71273) -- and more over in dotnet/runtimelabs; eg Initial hot cold splitting support for crossgen2/VM runtimelab#1900, Add Hot-Cold splitting experiment runtimelab#1915, ...
• Add GDV-like capability for delegate calls (Add support for delegate GDV and method-based vtable GDV #68703)
• Clone loops based on presence of loop-invariant GDV tests in the loop body (Loop cloning driven by type tests #65206) JIT: enable cloning based on loop invariant type tests #70377
• Above depends on: JIT: cloning invariant analysis seems oddly limited #70100 and Loop cloning misses definitions for fields of promoted structs #61040
• Fix OSR x64 epilog unwind (see note) JIT: revise approach for x64 OSR epilogs #65609
• Enable OSR by default for x64/arm64 Enable QJFL and OSR by default for x64 and arm64 #65675
• Address microbenchmark perf issues seen with QJFL=1 and OSR (see note) Revise heuristic for initial jitting. BenchmarkDotNet#1949 and Fix optimization of action methods for coreclr BenchmarkDotNet#1935
• JIT: Fold type checks with GDV #52370
• PGO: Optimize isinst with class probes #55325 (more generally: optimize type tests via PGO)

Under Consideration

• (EgorBo WIP for Native AOT) Add no-fallback GDV when runtime can provide full set of possible classes

Cut from .NET 7 Plans

• (EgorBo in PR) PGO: Add new tiers #70941
• Array interface method devirtualization #62457 (too many missing pieces)
• Fix issues in profile maintenance: JIT: PGO: fgOptimizeUncondBranchToSimpleCond corrupts profile data #50419, [JIT][PGO] Incorrect weights produced by expand-casts optimizations #50030, optOptimizeLayout introducing edge profile inconsistencies #48476
• PGO considerations for finally cloning #48925
• JIT: refactor edge weight representation #46885
• JIT: use normalized counts throughout #46883
• Reduce performance overhead of Tier0 instrumentation
• Add GDV-like capability for indirect calls (optimization of indirect calls #44610)
• JIT: optimize redundant type tests created by PGO #46887 (needs SSA update)

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue captures the planned work for .NET 7. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.

Summary

Dynamic PGO is an opt-in feature introduced in .NET 6. In .NET 7 we will work on making Dynamic PGO easier to use, applicable to more scenarios, and offering more performance improvements.

Planned for .NET 7

• Reduce performance overhead of Tier0 instrumentation
• Add GDV-like capability for delegate calls
• Add GDV-like capability for indirect calls (optimization of indirect calls #44610)
• Add support for enabling Dynamic PGO in SDK projects and runtime configs
• Clone loops based on presence of loop-invariant GDV tests in the loop body
• Add no-fallback GDV when runtime can provide full set of possible classes
• Array interface method devirtualization #62457
• JIT: optimize redundant type tests created by PGO #46887

Under Consideration

• PGO: Optimize isinst with class probes #55325 (more generally: optimize type tests via PGO)
• JIT: Fold type checks with GDV #52370
• Fix issues in profile maintenance: JIT: PGO: fgOptimizeUncondBranchToSimpleCond corrupts profile data #50419, [JIT][PGO] Incorrect weights produced by expand-casts optimizations #50030, optOptimizeLayout introducing edge profile inconsistencies #48476
• PGO considerations for finally cloning #48925
• JIT: refactor edge weight representation #46885
• JIT: use normalized counts throughout #46883
• Hot/Cold splitting

Author:	AndyAyersMS
Assignees:	-
Labels:	Epic, area-CodeGen-coreclr
Milestone:	7.0.0

[manofstick]

Hi!

...Sorry if this isn't the correct place for this post...

I've been playing around with PGO in .net 6 with mostly excellent results, but...

I've hit one case where it appears (to ignorant me) that the statistics collecting method isn't being replaced by the optimized version (well at least that is my simple understanding of what PGO should be doing...)

Anyway I'm not sure what information I can provide, I have tried creating a simple test case (with BenchmarkDotNet) but when it the simple case it appeared to do the "correct thing". i.e. works at the equivalent-ish speed.

But in my "production" case it consistently, without fail, always generates badness.

The method is generated using Expression trees - i.e. compiled Lambda - but is just a very simple object construction based on a BinaryReader, equivalent to the following c#:

SomeBusinessType ReadItem(BinaryReader reader)
{
    return new SomeBusinessType (
        reader.ReadUInt16(),
        CustomSerializationMethods.Methods.binaryReadCustomType(reader),
        reader.ReadSingle()
    );
}

The SomeBusinessType is an F# struct, but doesn't appear to be anything out of the ordinary.

It works perfectly fine and performant in "normal" .net 6, or with the PGO environmental variables set and debug build (although I assume that's just because it doesn't engage the PGO engine?).

As you can see in the image from the profiler, the compiled lambda, and a function JIT_ClassProfile32 completely dominate.

(It's called in a tight loop, populating an ImmutableArray<>.Builder, running on multiple threads - on Tasks started with Task.Run(). After a successful, albeit slow load, restarting this loading process, from within the same process, it still runs slowly - i.e. still not swapping out the function.)

Anyway, it's in proprietary code, and given that my simplified version didn't exhibit the same issue, I'm not sure what I can offer, but you can direct me to provide further information if you feel the need.

Edit: the code that generates the deserializer is used on numerous types - i.e standard reflection based Expression treee creation stuff. In the "production" run it seems to work fine for about a dozen Types, only running slowly for a single one. And noted above, even that single one, when done is an isolated project, worked as expected. Hmmm.

[AndyAyersMS]

...Sorry if this isn't the correct place for this post...

No worries, this is a good place.

What environment vars are you setting? If you've set COMPlus_TC_QuickJitForLoops=1 then it's possible for methods to get "trapped" in Tier0 code for long stretches of time, which would be consistent with what you're seeing above.

If so, the general fix for this is On Stack Replacement (aka OSR) which will be on by default in .NET 7 Preview 4.

There is an unsupported early version of OSR in .NET 6 which may or may not work... feel free give it a try by setting COMPlus_TC_OnStackReplacement=1.

You can also mitigate in source if you can restructure the code so that your loop is tiled into a two-level loop where the inner loop is in a helper method and the outer loop runs at least 30 iterations (split 10,000 into say -> 100 x 100). I realize this may not be easy if you don't have explicit looping.

[manofstick]

Yes indeed I had COMPlus_TC_QuickJitForLoops set, and setting COMPlus_TC_OnStackReplacement did indeed solve this. Thanks.

I did try splitting the loop, but couldn't seem to force it out of Tier0. I also looked at setting AggressiveOptimization on the method, but it was in f# and found that the f# compiler currently doesn't propagate that attribute to the generated IL, so that was a failure too.

Anyway, my main concern that this could of been an issue going forward has been alleviated, so thank for that.

[AndyAyersMS]

Work for .NET 7 is completed.