System.IO.IOException: The directory is not empty from TemporaryExeFile.Dispose()

[mikhail-tin]

Hi there!
We catch a lot of exceptions on a machine with slow HDD(Azure).
It's very simple to reproduce on the machine like we have in Azure, but impossible on local machine.
I assume the reason is OS can not free resources so fast.

System.IO.IOException: The directory is not empty.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.DeleteHelper(String fullPath, String userPath, Boolean recursive, Boolean throwOnTopLevelDirectoryNotFound, WIN32_FIND_DATA& data)
   at System.IO.Directory.Delete(String fullPath, String userPath, Boolean recursive, Boolean checkHost)
   at Medallion.Shell.Signals.WindowsProcessSignaler.**TemporaryExeFile.Dispose()**
   at Medallion.Shell.Signals.WindowsProcessSignaler.<TrySignalAsync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Medallion.Shell.Command.<>c__DisplayClass16_0.<<TrySignalAsync>g__TrySignalHelperAsync|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

I suggest using "try catch" and two attempts

            public void Dispose()
            {
                var toDelete = Interlocked.Exchange(ref this._path, null);
                if (toDelete != null)
                {
                   this.Cleanup(toDelete);
                }
            }

            private void Cleanup(string toDelete, bool firstAttempt = true)
            {
                try
                {
                    File.Delete(toDelete);
                    Directory.Delete(System.IO.Path.GetDirectoryName(toDelete));
                }
                catch (IOException)
                {
                    if (firstAttempt)
                    {
                        Task.Delay(0).GetAwaiter().GetResult();
                        this.Cleanup(toDelete, false);
                    }
                }
            }

Thx!

[madelson]

@mikhail-tin thank you for reporting!

This error is very interesting because it is failing to clean up a non-empty directory. I'm wondering how that directory could not be empty.

Is it possible for you to look at the file system and see what is in there? The temporary paths used by MedallionShell should look like Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")) .

Also, have you seen issues with Azure failing in this way before? Any links or information about them I could read?

Also, anything more you can tell me about your execution environment would be helpful. E. g. is this an Azure Windows Server VM? What OS version?

In the meantime, as a workaround it should be safe to do something like:

try { await command.TrySignalAsync(...); }
catch (IOException ex) { /* ignore */ }

[mikhail-tin]

Is it possible for you to look at the file system and see what is in there? The temporary paths used by MedallionShell should look like Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")) .

I have checked the directory based on Path.GetTempPath() and all folders are empty. So, It means the exe was deleted successfully.

Also, anything more you can tell me about your execution environment would be helpful. E. g. is this an Azure Windows Server VM? What OS

We run Windows Service use an GMSA on Azure VM with Windows Server 2012.

Also, I was trying to reproduce this issue use a simple console app, but didn't see any exception.

[madelson]

Found some additional evidence of this sort of behavior:

• https://superuser.com/questions/234569/windows-7-delayed-file-delete
• https://stackoverflow.com/questions/9370012/waiting-for-system-to-delete-file

[madelson]

@mikhail-tin thanks for the additional info. After doing some research on this, I think that the simplest fix may be to just avoid the temp directory altogether, and instead give the EXE a unique name when we generate it.

To that end, I've made a small change and published it in a prerelease version of the package (https://www.nuget.org/packages/MedallionShell/1.6.1-alpha01). Would you be able to test this out in your environment and verify whether this does indeed fix the issue for you?

If the fix works, I will publish this as a stable release.

For context, the relevant commit is here: 39bcc02

Thanks in advance for your help!

[mikhail-tin]

Thanks a lot! I don't think we can check new logic right now, due some changes in the environment. Anyway, we are going to remove "try catch" and only time will tell if there any issue.

and we can close the issue

[madelson]

@mikhail-tin thanks for the update. For my understanding, can you clarify what the result was here? Did the problem go away on its own? Did it get fixed by taking some other action (e. g. increasing disk speed)?

[mikhail-tin]

Hi, The issue was resolved by faster disk. Anyway, your solution should solve the issue. and add some perfomance improvments)