Umbrella: Replace IDynamicDocumentInfoProviders

[chsienki]

Today, razor tooling hosts the razor compiler in order to create design time documents. These are then injected into the Roslyn workspace via IDynamicDocumentInfoProvider.

Because the Roslyn workspace has the source generator, it also produces the same documents, and must be explicitly suppressed.

These means the compiler is used in two different ways: one for the tooling and one for the source generator. Ideally, we would just allow the source generator to run, and 'pull in' to the tooling the generated outputs. This would simplify the design of the compiler, allowing to potentially unlock performance enhancements.

Work items:

• Give Razor to access generator host outputs from Roslyn #8642
• Make RazorCodeDocument serializable #8643
• Implement LSP endpoints to request code document internals #8644
• Create a 'facade' code document that serializes on demand #8645
• Feature flag to switch been locally and remotely generated code documents #8646
• Use serialized code document internals inside tooling instead of generating locally #8647
• Stop injecting documents via IDynamicDocumentInfo and turn on generation in the generator #8648

IDE:

• Various IDE scenarios will stop working without document services #8845
• Add Using code action doesn't work in a world where Roslyn and Razor lack span mapping service #8671

[chsienki]

Link to internal doc: https://microsoft-my.sharepoint.com/:w:/p/chsienki/EeNFcrfDIlJNtpCd2Lrr4IEBlXhgOXorRawSyNaAlv1PLw?e=qfHpeJ

[davidwengier]

Added a couple of IDE issues too. Hope you don't mind, figured there was no point doing two tracking issues :)

[Mike-E-angelo]

Kindly checking in on this issue. My build times on my primary Blazor server-side csproj are officially over 30 seconds now when simply adding a new line to a .razor file 😭

[Mike-E-angelo]

Thank you for any update you can provide. Worth mentioning that it was only 17 seconds when reported in December.

So it would seem that I have incurred 15 seconds of additional build time over the course of 250 days with normal development, or .06 seconds per day.

[Mike-E-angelo]

Thank you for any update you can provide on this issue. To summarize my current development experience:

1. Building my Razor-based project takes roughly 32 seconds, up from 17 seconds when first reporting this over a year ago
2. The simplest keypresses will incur 10-70% CPU utilization for nearly 90 seconds, up from 10-20% for 25 seconds when first reporting this over a year ago

While 10-20% is easier to tolerate, the state of my environment now is becoming increasingly difficult to endure, and even more so the more that I develop.

Additionally, it is concerning that I appear to be the only person reporting such obvious challenges with your software. I again ask that if there is something in particular about my solution that is causing such grief that can be mitigated, please let me know so that I can take steps to reduce the friction.

Thank you again for your continued diligence and any assistance/updates you can provide.

[Mike-E-angelo]

Thank you for any update you can provide on this issue.

[Mike-E-angelo]

Fourty-second build time, from introducing a single new line in the Program.cs file:

Thank you for any update you can provide on this issue.

[Mike-E-angelo]

Thank you for any update you can provide on this issue. 🙏

This issue is now being tracked by my semi-weekly standups in Friction Points, where I discuss points of friction encountered in building a startup on Microsoft technology:
https://youtu.be/vmqPNZHDmVI?si=rp8W6AvJqRDmMpFN&t=601

[Mike-E-angelo]

This issue is being tracked by my semi-weekly standups in the Friction Points section, where I discuss points of friction encountered in building a startup based on Microsoft technology:

https://youtu.be/5vl6AIl-sNQ?si=MfXVwty5FO0j-WWd&t=719

Thank you for any update on this issue that you can provide. 🙏

[Mike-E-angelo]

Sounds like this issue's status is at "we're aware of it and we'll talk about it at some point. maybe."

#9369 (comment)

[Mike-E-angelo]

This issue is being tracked by my semi-weekly standups in the Friction Points section, where I discuss points of friction encountered while building a startup based on Microsoft technology:

https://youtu.be/Ho4uhaow0ms?si=0g1pFFXn_9teKdbG&t=527

[Mike-E-angelo]

Wow, so good news here... I tried this with the latest preview, and build times have been reduced by half. 😊 Maybe a side effect of #9214? 🤔

Whatever the reason, I am now seeing ~20-second build times whereas before I was seeing 40+ seconds:

Thank you for all your efforts out there! Happy Holidays. ❄

[Mike-E-angelo]

FWIW/FYI I continue to track this issue in our semi-weekly standups:
https://youtu.be/dLh5S1DcNFg?si=my0iX0HJYGm-dVzN&t=591

Thank you for any consideration toward improving the experience of a startup using your technology to succeed. 🙏

🦌 Happy Holidays ⛄❄

[Mike-E-angelo]

Happy New Year out there 🙏

FWIW/FYI I am still tracking this issue and made mention of it today on X:
https://twitter.com/Mike_E_angelo/status/1755588012962963748

[davidwengier]

issue has been known for many months now and has not even started work yet. :(

The good news is that we're all much better at writing code than we are at issue tracking, project management and keeping GitHub issues up to date. I had a quick look and about 80% of our recent PRs are part of the broader work to move off IDynamicFileInfo. If you ever see terms like "cohosting" or project "fuse", those are all part of it too. We even have weekly meetings about it :)

[Mike-E-angelo]

The good news is that we're all much better at writing code than we are at issue tracking, project management and keeping GitHub issues up to date

That really is good news, @davidwengier! Thank you very much for the update. That is awesome to hear that there's been work done towards this goal, along with the oh-so-amazing news that #8371 has been addressed too. 🙏🙏🙏

I've subscribed to the issues you have mentioned and will be following along. Thank you again for all your efforts out there. This has been a great week for me to have learned all this. 😌 👍

[Mike-E-angelo]

FWIW I made some major project reductions in my solution and shaved off about 10 seconds of compile+launch time:
https://twitter.com/Mike_E_angelo/status/1762043844743368953

😌

Unfortunately, however, this issue is creeping up with the build times again, going from 20 seconds the last I checked, to about 24-25 now. 😭

[Mike-E-angelo]

Back again to 35-40 second build times with one line of code change in the project. 😞

It's a real drag, team. Please send halp. 😞😞😞

[davidwengier]

If the build times are slow for regular builds, and not hot reload/enc related, then IDynamicDocumentInfo is not in play. I would recommend running dotnet build -bl to get a binary log of the build and see where the time is being spent. It's worth noting that the log file can contain information you might not want to be public (project file contents, file names and paths of every source file, etc.) so probably best to report through VS Feedback, that way the attachment is secure.

[Mike-E-angelo]

Thank you for the update @davidwengier. If you can verify this ticket, it would be greatly appreciated. It's the reason why I have been pinging here. :)

https://developercommunity.visualstudio.com/t/Building-After-a-Few-Simple-Keypresses-T/10230756

As you can see, it's pointing to this issue.

[Mike-E-angelo]

Updated with Version 17.10.0 Preview 4.0 build + binlog here:

https://developercommunity.visualstudio.com/t/1-Line-of-Code-Takes-Nearly-40-Seconds-t/10642151

[Mike-E-angelo]

Hi @davidwengier any luck with the above, by chance? Thank you for efforts out there. 🙏

[davidwengier]

I was just trying to direct you to the right place I'm afraid. The issue you logged is in the Razor compiler teams area and I'm sure they'll update it as appropriate. I doubt I'd be any help if I looked at it :)

[Mike-E-angelo]

Whoa, so my original issue was closed and inadvertently redirected to the wrong issue here, do I have this understood correctly? 😭 I have been pinging this thread for over 8 months now, thinking this was the place to update as that is what the now-closed devcom ticket stated. 😞 All the while facing increasing build times as my codebase expands... 💔

[davidwengier]

I was refering to the issue in your comment above mine, developercommunity.visualstudio.com/t/1-Line-of-Code-Takes-Nearly-40-Seconds-t/10642151. That issue is not closed.

[Mike-E-angelo]

That issue is not closed

Correct, I understand. There are two issues. The one that I reported on December 14th, 2022 -- 498 days ago -- was closed and directed to this issue, which now appears to be in error. As this is now the biggest source of grief I am encountering with Visual Studio, I hope my concern is understandable.

[davidwengier]

which now appears to be in error.

I'm not sure I'd agree with that, there are comments from people looking at memory dumps and traces and identifying issues, and it is entirely reasonable for IDynamicFile to be a source of perf concerns in the IDE, and there also being perf issues during build.

[Mike-E-angelo]

and there also being perf issues during build

So, two issues then? 😉

[davidwengier]

If only there were only two issues

[Mike-E-angelo]

And now there are. ✨ My beef is that it took nearly 1.5 years to get to them. :)

[davidwengier]

I've just realised why a build slowness issue could be duped here, sorry for not thinking of this earlier. It could be that the cause of the slowness was determined to be doing the work a source generator needs to do to discover things about your projects, which it then caches for future runs. That info is also present in the IDE, but due to how its architected to use IDynamicFile etc., those caches are separate. The work going on now is to combine those two worlds into one to take advantage of the perf benefits.

However if builds have been getting slower over time for no discernable reason, its worth the new issue being created so the compiler team can at least check and see if there is something else, or something new, going on.

[Mike-E-angelo]

Sounds good to me, @davidwengier. Thank you for your efforts out there and for dealing with all the perpetually moving pieces. 👍🙏

[Mike-E-angelo]

FWIW/FYI: https://www.reddit.com/r/Blazor/comments/1cu0lh2/anyone_else_with_large_blazor_solutions_do_you/

[Mike-E-angelo]

Would it be possible to hear what exactly is causing this issue @davidwengier? It would be valuable. I got a suggestion here about moving code into .cs code-behind and wonder if that would improve this? 🤔 That would really suck because I have so many files, though. 😅

https://www.reddit.com/r/Blazor/comments/1cu0lh2/comment/l4g0pdl/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Using RCL is also suggested and I am wondering if that would help here. 🤔
https://www.reddit.com/r/Blazor/comments/1cu0lh2/comment/l4g5phk/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button <-- Nevermind, I am already using these 😞

[Mike-E-angelo]

@davidwengier / @phil-allen-msft:
https://developercommunity.visualstudio.com/t/Changing-1-Line-of-Code-Takes-Nearly-40/10642151#T-N10700341

🤩🤩🤩
🙏🙏🙏
😌😌😌

Thank you thank you thank you!!!

Believe it or not, that is the last remaining/identified performance issue in my world. I am so happy I cannot express how much joy I have at the moment. Thank you again for all your diligence in improving the quality of your software. It means everything!!!

[Mike-E-angelo]

https://www.reddit.com/r/Blazor/comments/1e24zag/long_build_times_try_this_thank_you_blazor_razor/

🙏

[ScarletKuro]

https://www.reddit.com/r/Blazor/comments/1e24zag/long_build_times_try_this_thank_you_blazor_razor/

🙏

Can someone from the Razor team confirm if this feature will be shipped with the .NET 9 release and if it will be enabled by default?

[Pinox]

Hi @Mike-E-angelo

On reddit I see you use cloc.

Never heard of it and it looks good ;)) What would the command be to run this on a visual studio solution or project folder. I need a shorter cheat sheet than the very comprehensive documentation ;))

[Mike-E-angelo]

Hey @Pinox indeed it took me a bit to figure it out. This is what I use from the SLN root:

cloc . --exclude-dir=_ReSharper.Caches,Generated,Migrations,.git,.vs,DragonSpark.Application.Testing,DragonSpark.Testing,Starbeam.Testing --include-lang=C#,Razor

As you can see I do not include any testing assemblies in my totals. Replace with yours and you should be good to go. 👍