Fix Sqlite database locking issue

[Zeegaan]

Issue

• When saving 2 document types (with templates) at the same time, the SqlLite database would totally lock up, and thus the whole application would be unusable.
  This happens because when we save a document type, it needs to also create a template file on disk, thus it calls the FileService CreateTemplateForContentType, this file service would create a scope, get a read & write lock, and then call the TemplateRepository, that then would insert the template both in the database, but also create the file on disk.
  All while the other thread would do the exact same steps as above, which would then result in the complete locking of the database. Why this results in the complete locking of the database I'm not sure, but it seems like both are taking writelocks, locking the database, but now its locked so neither can write.

Here is me replicating the issue:

Notes

• I have implemented a lock, that prevents simultaneous creation of these scopes in the FileService, which prevents the issue.
• This could also be solved by changing the IsolationLevel for the scope, so it would be using ICoreScope scope = ScopeProvider.CreateCoreScope(IsolationLevel.UnCommited); but this changed how strictly the database handles locking.

How to test

• Run a fresh umbraco install in debug
• Set a breakpoint in ContentTypeController in the PostSave action on line 319.
  
• Open backoffice in 2 tabs and create a document type in each (do not press save yet)
• When both is ready, save the first (it should hit the breakpoint), when the breakpoint is hit, save the other.
• Now both should be caught on the breakpoint which ensures they will run simultaneously.
• Remove the breakpoint and run the application (F5), this should no longer result in the locking of the database, and thus both document types should be saved 👍

[ronaldbarendse]

@Zeegaan This is probably only an issue in SQLite because that allows at most one writer to proceed concurrently . Both FileService and ContentTypeService will create separate write transactions, but the parent transaction is only committed at the end of the request, causing them to deadlock. A better solution might be to create a single scope that wraps both service calls, so there's only a single scope that takes a write lock...

[Zeegaan]

As @ronaldbarendse mentioned there is even more solutions to this problem, this will need an internal discussion before getting merged 👍

[matthewcare]

@Zeegaan
I found this issue when implementing batch media uploads (#12947)
I added Semaphore to the API to only process on a single thread at a time, effectively queuing the requests.
Still waiting back from HQ about if that's the preferred method, but it seems like whatever the answer is is how this should also be handled?

[Zeegaan]

@matthewcare We will have a discussion monday on how to go about it, but I agree, we should probably handle both the same way, thanks for letting me know! 🙌

[ronaldbarendse]

This must be one of the deadlock issues mentioned in dotnet/efcore#28172 (comment), as Umbraco opts-in to this 'pit-of-failure' 🤔