JIT: initial implementation of profile synthesis

[AndyAyersMS]

Implements a profile synthesis algorithm based on the classic Wu-Larus paper Static branch frequency and program profile analysis (Micro-27, 1994), with a simple set of heuristics.

First step is construction of a depth-first spanning tree (DFST) for the flowgraph, and corresponding reverse postorder (RPO). Together these drive loop recognition; currently we only recognize reducible loops. We use DFST (non-) ancestry as a proxy for (non-) domination: the dominator of a node is required to be a DFST ancestor. So no explicit dominance computation is needed. Irreducible loops are noted but ignored. Loop features like entry, back, and exit edges, body sets, and nesting are computed and saved.

Next step is assignment of edge likelihoods. Here we use some simple local heuristics based on loop structure, returns, and throws. A final heuristic gives slight preference to conditional branches that fall through to the next IL offset.

After that we use loop nesting to compute the "cyclic probability" $cp$ for each loop, working inner to outer in loops and RPO within loops. $cp$ summarizes the effect of flow through the loop and around loop back edges. We cap $cp$ at no more than 1000. When finding $cp$ for outer loops we use $cp$ for inner loops.

Once all $cp$ values are known, we assign "external" input weights to method and EH entry points, and then a final RPO walk computes the expected weight of each block (and, via edge likelihoods, each edge).

We use the existing DFS code to build the DFST and the RPO, augmented by some fixes to ensure all blocks (even ones in isolated cycles) get numbered.

This initial version is intended to establish the right functionality, enable wider correctness testing, and to provide a foundation for refinement of the heuristics. It is not yet as efficient as it could be.

The loop recognition and recording done here overlaps with similar code elsewhere in the JIT. The version here is structural and not sensitive to IL patterns, so is arguably more general and I believe a good deal simpler than the lexically driven recognition we use for the current loop optimizer. I aspire to reconcile this somehow in future work.

All this is disabled by default; a new config option either enables using synthesis to set block weights for all root methods or just those without PGO data.

Synthesis for inlinees is not yet enabled; progress here is blocked by #82755.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Implements a profile synthesis algorithm based on the classic Wu-Larus paper (Static branch frequency and program profile analysis, Micro-27, 1994), with a simple set of heuristics.

First step is construction of a depth-first spanning tree (DFST) for the flowgraph, and corresponding reverse postorder (RPO). Together these drive loop recognition; currently we only recognize reducible loops. We use DFST (non-) ancestry as a proxy for (non-) domination: the dominator of a node is required to be a DFST ancestor. So no explicit dominance computation is needed. Irreducible loops are noted but ignored. Loop features like entry, back, and exit edges, body sets, and nesting are computed and saved.

Next step is assignment of edge likelihoods. Here we use some simple local heuristics based on loop structure, returns, and throws. A final heuristic gives slight preference to conditional branches that fall through to the next IL offset.

After that we use loop nesting to compute the "cyclic probability" $cp$ for each loop, working inner to outer in loops and RPO within loops. $cp$ summarizes the effect of flow through the loop and around loop back edges. We cap $cp$ at no more than 1000. When finding $cp$ for outer loops we use $cp$ for inner loops.

Once all $cp$ values are known, we assign "external" input weights to method and EH entry points, and then a final RPO walk computes the expected weight of each block (and, via edge likelihoods, each edge).

We use the existing DFS code to build the DFST and the RPO, augmented by some fixes to ensure all blocks (even ones in isolated cycles) get numbered.

This initial version is intended to establish the right functionality, enable wider correctness testing, and to provide a foundation for refinement of the heuristics. It is not yet as efficient as it could be.

The loop recognition and recording done here overlaps with similar code elsewhere in the JIT. The version here is structural and not sensitive to IL patterns, so is arguably more general and I believe a good deal simpler than the lexically driven recognition we use for the current loop optimizer. I aspire to reconcile this somehow in future work.

All this is disabled by default; a new config option either enables using synthesis to set block weights for all root methods or just those without PGO data.

Synthesis for inlinees is not yet enabled; progress here is blocked by #82755.

Author:	AndyAyersMS
Assignees:	AndyAyersMS
Labels:	area-CodeGen-coreclr
Milestone:	-

[AndyAyersMS]

@BruceForstall PTAL
cc @dotnet/jit-contrib

Was about to post a sample result but realized to my dismay the final weight computation is off. So will fix that and then put up some examples.

[AndyAyersMS]

Was about to post a sample result but realized to my dismay the final weight computation is off. So will fix that and then put up some examples.

This looks like a bug in the DFS traversal (which is surprising since we rely on it for many a few other things). Given the following graph, it produces the DFST shown in blue. Each node is annotated with the preorder and postorder numbers.

The RPO from this is

01 -> BB01[1, 9]
02 -> BB06[2, 8]
03 -> BB07[7, 7]
04 -> BB08[9, 6]
05 -> BB09[8, 5]
06 -> BB02[3, 4]
07 -> BB03[4, 3]
08 -> BB04[5, 2]
09 -> BB05[6, 1]

Note that BB7 appears too early in the RPO, it should appear after both its pred BB6 and its pred BB4. So when we do our final RPO walk to set profile counts, we don't include the count from BB4.

The issue seems to be that when we initially visit BB6 we push BB7 onto the pending visit stack and mark it, so when we reach BB4 we don't visit BB7.

[AndyAyersMS]

This looks like a bug in the DFS traversal (which is surprising since we rely on it for many things)

I have a fix I will PR separately. Somewhat surprisingly it is no diff (*). What we were doing produces a spanning tree, but not a proper depth first spanning tree. Looks like existing uses are somehow ok with that.

With things fixed we get

After doing a post order traversal of the BB graph, this is the ordering:
01 -> BB01[1, 9]
02 -> BB06[2, 8]
03 -> BB02[6, 7]
04 -> BB03[7, 6]
05 -> BB04[8, 5]
06 -> BB05[9, 4]
07 -> BB07[3, 3]
08 -> BB08[4, 2]
09 -> BB09[5, 1]

and now we see BB07 properly comes after BB06 and BB04.

Resulting synthesized profile is:

(*) caveat -- the method above fails in SPMI during importation because of missing info, so perhaps there would be diffs on a clean collection?

[BruceForstall]

Suggested change

	// 2: profile synthesis for foot methods w/o pgo data
	// 2: profile synthesis for root methods w/o pgo data

[BruceForstall]

I think you need to add fgprofilesynthesis.h in the JIT_HEADERS section, below

[AndyAyersMS]

added

[BruceForstall]

Should ProfileSynthesis::Run() set this instead?

[AndyAyersMS]

Yeah, that makes more sense.

[AndyAyersMS]

Over time this flag will either change into something more complex or go away.

[BruceForstall]

Suggested change

	// entrely on heuristics.
	// entirely on heuristics.

[BruceForstall]

Is implies a bool return. Maybe GetLoopFromHeaderBlock?

[AndyAyersMS]

Sure

[BruceForstall]

Suggested change

	// AssignLikelihoodCond: update edge likelihood for a block that
	// AssignLikelihoodSwitch: update edge likelihood for a block that

[BruceForstall]

I presume we have no degenerate (n == 0) cases here?

[AndyAyersMS]

Seems like we don't? Even if a switch is just default it has 1 case. I'll add an assert for now.

[BruceForstall]

Suggested change

	const unsigned n = block->NumSucc();
	const unsigned n = block->NumSucc(m_comp);

?

[AndyAyersMS]

This one is intentional -- if there are $n$ switch cases then each case gets $1/n$ of the flow. In the subsequent loop we just look at targets and give each target edge a likelihood of $dupCount/n$.

[BruceForstall]

Ok, makes sense. Might want to add a comment to ensure that is perfectly clear, since we normally pair NumSucc()/GetSucc()/Succs() and NumSucc(Compiler*)/GetSucc(Compiler*,x)/Succs(Compiler*) and seeing them "unbalanced" is weird.

[BruceForstall]

Wouldn't all the heuristics in AssignLikelihoodCond apply? And need to get blended somehow?

[AndyAyersMS]

Probably something like that. I kept the heuristics fairly simple for now. Plan is to revise them once we start comparing the synthesized results to real profile data.

[BruceForstall]

Suggested change

	for (BasicBlock* const succBlock : loopBlock->Succs())
	for (BasicBlock* const succBlock : loopBlock->Succs(m_comp))

?

[AndyAyersMS]

Fixed

[BruceForstall]

When you assign likelihoods from the heuristics, you have a fixed ordering of heuristics (and fixed likelihoods), and the first heuristic that matches determines the resulting likelihoods. In Wu/Larus, if I read it correctly, they combine likelihoods from all applicable heuristics. Is that something that should be done?

[AndyAyersMS]

When you assign likelihoods from the heuristics, you have a fixed ordering of heuristics (and fixed likelihoods), and the first heuristic that matches determines the resulting likelihoods. In Wu/Larus, if I read it correctly, they combine likelihoods from all applicable heuristics. Is that something that should be done?

My main goal here is to get the solver framework in place; to me, that's the interesting part of the change.

The heuristics will evolve, but whether they end up looking like they do now, or more like what's in the paper, or something else remains to be determined. For instance, we're currently running before importation so (for the most part) can't use any heuristic that relies on knowing the computations in each block.

This is somewhat intentional, as the importer is sensitive to profile data, so we need some notion of profile beforehand, but also somewhat unfortunate, because once we start importing we'll likely uncover things that would lead us to make different likelihood projections. I haven't yet pinned down when or how often we might revisit / rerun synthesis.