optimize: add at least a small cost for most instructions

[vtjnash]

This should better reflect the aggregate expected cost of their presence
and restrict endless inlining: SROA may eliminate many of them, but
probably not all of them.

This was part of #31338, but I realized that I need to separate the nanosoldier results.

[vtjnash]

@nanosoldier runbenchmarks(ALL, vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[timholy]

Looks like some regressions in array handling. Maybe pair this with an increase in the cost thresholds?

[vtjnash]

That would somewhat defeat the purpose of this. In the benchmark, it looks like this improved performance by 10% on many benchmarks, and up to 40% on others. I would rather understand why that one benchmark is not being optimized well.

@nanosoldier runbenchmarks(ALL, vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[timholy]

That would somewhat defeat the purpose of this.

Depends on the purpose; it wouldn't defeat the "endless inlining" (in the sense of infinite). But agreed, best to focus on the few failures.

[vtjnash]

@nanosoldier runbenchmarks("broadcast", vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[vtjnash]

Results seem pretty good to me now, so I'll plan to merge over the weekend, unless there's any more comments.

[KristofferC]

Why did you only run the broadcast benchmarks? There where categories with regression outside of that.

ID	time ratio	memory ratio
["array", "index", "("sumvector_view", "1:100000")"]	1.03 (50%)	1.40 (1%) ❌
["array", "index", "("sumvector_view", "BitArray{2}")"]	1.60 (50%) ❌	2.49 (1%) ❌
["array", "index", "("sumvector_view", "SubArray{Float32,2,Base.Res...},true}")"]	2.26 (50%) ❌	2.49 (1%) ❌
["array", "index", "("sumvector_view", "SubArray{Int32,2,Base.ReshapedArray{Int...true}")"]	2.23 (50%) ❌	2.49 (1%) ❌

for example

[vtjnash]

Those are the only ones that would have changed. I looked into several others. Most were expected/intended. The usual causes were an insufficient benchmark complexity, that's now just measuring overhead noise (e.g. scalar iteration), and inaccurate cost modeling issues (harder to deal with, but maybe we should store the result for boundscheck=none at least? this problem existed before, we just don't have good benchmark coverage of the case.)

[mbauman]

View construction and indexing seems like quite a relevant microbenchmark... and is something we've tried hard to make as fast as possible. Why can't we just manually inline whatever is throwing a wrench into things here?

Edit: oh, without looking I'd bet it's the generic first that's not getting inlined. Alright, if that's the case I'm slightly more sympathetic.

[vtjnash]

@nanosoldier runbenchmarks(ALL, vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[vtjnash]

Wow, okay, that seems quite bad

[vtjnash]

@nanosoldier runbenchmarks(ALL, vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @christopher-dG

[vtjnash]

Not sure that it is worthwhile trying to figure out those regressions for this minor gain