Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VsChromium search stops working after native find-in-files is performed in VS2019 16.5.2 #61

Closed
kayru opened this issue Apr 8, 2020 · 22 comments
Assignees

Comments

@kayru
Copy link

kayru commented Apr 8, 2020

Another issue with the same project setup as #60 (VsChromium-0.9.31, UE4 codebase with custom project config). Code search works until native visual studio find-in-files operation is done via ctrl+shift+f. All functionality stops working after that. Code search does not seem to perform any operations and opening VsChromium Indexing Server Information window gets stuck on "Please Wait" message.

Please let me know if there are any additional diagnostics / logs that I could provide to help you reproduce and investigate the issue.

@kayru
Copy link
Author

kayru commented Apr 8, 2020

Just confirmed that this issue reproduces in other projects too, so seems like there is nothing specific to UE4.

@rpaquay
Copy link
Contributor

rpaquay commented May 27, 2020

@kayru

Does this still happen?

One thing that could help is to attach the log files to this bug, just to see if there is a meaningful error logged in there. The log files are located in "%LOCALAPPDATA%\VsChromium". Up to 10 files of 2MB files each are use for logging.

VsChromium.log is used for information/warning/error level logging.
VsChromium.errors.log is used for warning/error level logging only.

@torlack
Copy link

torlack commented May 29, 2020

I'm seeing this very issue now. While I can't include the log files due to sensitive information, I can explain what I am seeing.

First, there are no errors in the error log file other that one regarding a locked file in the source tree that we get during solution load.

After solution load, nothing of interest is in the log file. After I do a VsChr search, I can see the search results in the log file (229 lines of log).

With in a few seconds after I do a VS find in files, the log file starts growing rapidly. The file contains a massive series of "Register path", "Unregister path", "FlushFileRegistrationQueueTask", "Request # of type RegisterFileRequest", and "Request # of type UnregisterFileRequest" log lines.

It appears to be registering and then un-registering almost every file in the solution.

By the time I force quit VS, the VsChr log file was nearly 8000 lines long with this spam. 1921 requests were in the log file.

@rpaquay
Copy link
Contributor

rpaquay commented May 29, 2020

I see, this is very useful. It is likely that VS is behaving this way for some odd reason related to your configuration and/or solution, but it is clearly an issue with VsChromium, which should be able to deal with this amount of changes.

Would you mind trying the v0.33 pre-release (https://github.com/chromium/vs-chromium/releases/tag/v0.9.33)? It does have an improvement to this kind of VS behavior, so maybe this will address your problem while I investigate the root cause of this issue.

@torlack
Copy link

torlack commented May 29, 2020

It didn't help.

If you need me for anything, let me know. Love the tool.

@kayru
Copy link
Author

kayru commented May 31, 2020

Yes, the issue is still present.

Installed v0.33 pre-release. Opened a relatively small project, confirmed vs-chromium search works via ctrl+shift+;, invoked native find-in-files via ctrl+shift+f, confirmed that vs-chromium becomes broken/stuck as before. Logs: https://gist.github.com/kayru/1f6cf8b4a1a3c74d6eaf7ee72a4b7f5b

@rpaquay
Copy link
Contributor

rpaquay commented Jun 2, 2020

Thank you @kayru, I am able to reproduce the issue. I am working on a fix.

Fyi, the line of code causing the bug is https://github.com/chromium/vs-chromium/blob/master/src/Server/Threads/ThreadObject.cs#L46

In some case, the _currentTask field is reset to null after a new task has been scheduled by another thread. Multi-threading is hard...

@rpaquay rpaquay self-assigned this Jun 3, 2020
rpaquay added a commit that referenced this issue Jun 9, 2020
Instead of implementing a custom queue with custom locks, CustomThreadPool
now uses a single BlockingCollection to add elements, and a fixed set
of threads consuming elements concurrently using GetConsumingEnumerable.

This fixes bug #60 and #61, at the expense of losing the ability of shutting
down idle threads in the pool. We decreased the pool size from 10 to 5.
@rpaquay
Copy link
Contributor

rpaquay commented Jun 9, 2020

@rpaquay
Copy link
Contributor

rpaquay commented Jun 10, 2020

@kayru There is a new pre-release (0.9.34) that should fix all issues related to index not working, etc. after performing a "Find in Files". If you have time to try it out, please let me know if it did indeed fix the problems you were seeing, so that I can make this pre-release an "official" release.

@rpaquay
Copy link
Contributor

rpaquay commented Jun 10, 2020

@torlack Same comment: There is a new pre-release (0.9.34) that should fix all issues related to index not working, etc. after performing a "Find in Files". If you have time to try it out, please let me know if it did indeed fix the problems you were seeing, so that I can make this pre-release an "official" release.

@kayru
Copy link
Author

kayru commented Jun 11, 2020

@rpaquay Unfortunately it does not look like the fix worked. As before, the the indexing server information window stops working as soon as built-in search starts and code search queries don't work.

Edit: curiously the issue is not reproducible in a tiny project, only in our main big UE4 project.

@rpaquay
Copy link
Contributor

rpaquay commented Jun 11, 2020

@kayru Sorry about that, I thought I had fixed the root cause, as this was fixed in the small project I was using to reproduce. Do you see errors in the error log file? Anything you can share?

@kayru
Copy link
Author

kayru commented Jun 11, 2020

Nope, no errors in the logs. I also noticed that in a "medium sized" project, vs-chromium hangs but eventually recovers. I will try to see if the issue is reproducible in the GitHub version of UE4, in which case you may be able to just use that.

@rpaquay
Copy link
Contributor

rpaquay commented Jun 11, 2020

@kayru

Oh, it is interesting it eventually recovers on a medium project. It could be that there are so many open/close files requests that VsChromium is temporarily overwhelmed and becomes "unresponsive" for a while, i.e. the new requests are still technically processed, but a while later after VsChromium had time to catch up. In the case of your big UE4 project, it could be that it takes minutes for VsChromium to catch up.

On your big UE4 project, when VsChromium is "unresponsive", do you see spam activity in the regular log file for a long time, or do you see no activity at all?

If it is the case that VsChromium is jus overwhelmed with too many requests, then I will have to work on an additional fix to address that.

@rpaquay
Copy link
Contributor

rpaquay commented Jun 11, 2020

I was able to reproduce the behavior mentioned in a previous comment. When "Find in Files" goes through hundred of thousand of source files, VsChromium is overwhelmed with File/Close requests, and takes more than 30 min to catch up.

  • In version 0.33 and earlier, there were issue with multi-threading on the server, and VsChromium ended up in a state where it would never recover

  • In version 0.34, VsChromium eventually recovers, but it takes 30+ min in some cases

(It is somewhat amusing how VsChromium ends up showing search results after 30 min or so :))

rpaquay added a commit that referenced this issue Jun 12, 2020
Instead of using the ITextDocumentFactoryService component, we now use
IVsRunningDocumentTable to keep track of when files are open and closed
in the VS editor.

The main reason is that ITextDocumentFactoryService keeps track of all
ITextBuffer instances used in VS, including ITextDocument instances
loaded for temporary usage, such as the "Find in File" features.

This change ensures the number of events related to opening and closing
documents is close to zero when using "Find in Files" on large directories
or project.

This is (hopefully) the last part to fix issue #61
@rpaquay
Copy link
Contributor

rpaquay commented Jun 12, 2020

@kayru

The issue with large projects should be fixed in release 0.9.35: https://github.com/chromium/vs-chromium/releases/tag/v0.9.35

@kayru
Copy link
Author

kayru commented Jun 13, 2020

Confirmed. Seems to work with UE4 solution now. Thank you!
I'll test this build for the few days before knowing for sure it's robust, but looks promising :)

@kayru
Copy link
Author

kayru commented Jun 20, 2020

I'm happy to confirm that 0.9.35 has been solid for me during daily use for the past week. Ship it :)

@rpaquay
Copy link
Contributor

rpaquay commented Jun 22, 2020

Great. I will release it this week.

@torlack
Copy link

torlack commented Jun 22, 2020

I finally had a chance to test the fix and can confirm it works on a Frostbite game. So that is two massive game solutions known to work now :D

@rpaquay
Copy link
Contributor

rpaquay commented Jun 22, 2020

@torlack Thank you for confirming.

Fyi, I just released 0.9.35, but I noticed issue #69 (VsChromium does not show a banner when a new update is available), so I am going to make a new release shortly to fix that issue too. This issue does not affect you since you already have the latest version (the binary of the 0.9.35 release is the same binary you have been using), but it will affect you in the future, as you won't get a banner in Code Search when a new version of VsChromium is available.

@rpaquay
Copy link
Contributor

rpaquay commented Jun 22, 2020

Ok, I released version 0.9.36, which solves the "no update banner" issue (that will show up when the next release is available of course). I am closing this bug as fixed.

@rpaquay rpaquay closed this as completed Jun 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants