Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WSL2: Virtual hard disk files must be uncompressed and unencrypted and must not be sparse #4103

Open
squeaky-pl opened this issue Jun 13, 2019 · 86 comments
Assignees
Labels
bug failure-to-launch failure to launch workaround-available wsl2 Issue/feature applies to WSL 2

Comments

@squeaky-pl
Copy link

Please fill out the below information:

  • Your Windows build number: Microsoft Windows [Version 10.0.18917.1000]

  • What you're doing and what's happening:
    I am trying to launch new distro under WSL2. I did first wsl --set-default-version 2, went to the shop and downloaded Ubuntu. Then I went to Start and clicked Ubuntu. After couple of seconds I was greeted with following error:

Screenshot_205

  • What's wrong / what should be happening instead:

I should get a brand new distro under WSL2.

@squeaky-pl
Copy link
Author

Attaching logs.

logs.zip

@squeaky-pl
Copy link
Author

OK, I managed to resolve it by myself after some intensive Googling by running those two commands:

fsutil behavior set disableencryption 1
fsutil behavior set disablecompression 1

and rebooting.

Now it just works™

Nonetheless I dont remember changing anything about NTFS behavior myself. Could it be that my laptop manufacturer (Dell) did it? Maybe this should be handled more gracefully by WSL2. Could WSL2 set those properties per virtual hard disk file to prevent others from hitting this?

@benhillis
Copy link
Member

Thank you very much for filing this issue. It's interesting that I'm not seeing that error in the logs, I'll have to dig into why that isn't showing up. I am seeing an invalid parameter error when trying to mount the VHD, so I bet that's the inner error.

When creating the VHD's I'll make sure that compression and encryption are both disabled on them.

@benhillis benhillis self-assigned this Jun 13, 2019
@benhillis benhillis added bug wsl2 Issue/feature applies to WSL 2 workaround-available labels Jun 13, 2019
@caffme
Copy link

caffme commented Jun 13, 2019

I can not update my existing distro, I am getting the same error in PS when trying to run the wsl --set-version. The workaround (set disableencryption and disablecompression, followed by reboot) did not help.

I can install a different distro and that works. I only tried that after I tried the workaround.

shot_190613_171648

wsllogs.zip

@caffme
Copy link

caffme commented Jun 13, 2019

Also, after hitting the error subsequent attempts give a different error: "The operation timed out because a response was not received from the virtual machine or container." unless I use hcsdiag to kill the wsl container.

shot_190613_181110

@dmchurch
Copy link

I'm encountering the same issue as @caffme. I assume this is because the folder the VHD is supposed to go into is marked as compressed or something, even though I've done the fsutil workaround. Where is the WSL2 VHD supposed to be located? I can't find any documentation on that, even though the "expand your VHD" documents mention putting in the path to it.

@dmchurch
Copy link

Okay, after some lucky guesses I've confirmed it and I have an addendum to the workaround (cc @caffme in case this works for you as well) - go to the Ubuntu directory in your profile folder (something like %USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents" (and I suppose "Encrypt contents" as well if that's checked). When it asks whether you want to apply to just this folder or to all subfolders and files, you can say "just this folder", because all you're doing is clearing that "compress" flag. After that the wsl --set-version command should work.

@illera88
Copy link

I had the same error. Fixed with a combination of @squeaky-pl and @dmchurch solutions

@caffme
Copy link

caffme commented Jun 14, 2019

@dmchurch Thanks, yes that worked for me. I wondered about checking that but had no idea where WSL stored anything, good hunting!

@benhillis I am fairly sure that I didn't set the compressed flag myself. C:\Users\James\AppData\Local\Packages was the highest folder in the tree that had the compressed flag and I would not have done that manually.

@krzys-h
Copy link

krzys-h commented Jun 22, 2019

Thanks, the workaround of manually disabling compression on %localappdata%\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc worked for me. My guess is that the compression was enabled by the update that started doing that automatically a while ago to free up space for more updates (except from my experience it never actually decompresses them afterwards, unlike what the official website says)

@jsbarrett
Copy link

Just wanted to drop in and say thanks to @dmchurch for the tip about uncompressing those folders (and everyone else in the discussion helping each other find answers)

And afterwards I got the error about timing out ... so I just restarted the computer and tried again and it worked just fine.

@WSLUser
Copy link

WSLUser commented Jul 3, 2019

When creating the VHD's I'll make sure that compression and encryption are both disabled on them.

Shouldn't the more correct solution be to support compression/decompression and encryption/decryption like you would with any other VHD utility? I'd say disabling is a good hot fix, but will need revisiting.

@benhillis
Copy link
Member

@WSLUser - Indeed, we are looking into better solutions.

@mebeZ
Copy link

mebeZ commented Aug 8, 2019

Here's what worked for me...

Go to this link: https://docs.microsoft.com/en-us/windows/wsl/install-win10

When you're installing linux distro, do option 3: Download and manually unpack and install (for Windows Server - instructions here)

@scardena
Copy link

I tried @dmchurch and @squeaky-pl solutions, and although I got rid of the "The requested operation could not be completed due to a virtual disk system limitation", now I am getting:

Exporting the distribution failed.
bsdtar: Write error

@kevinthenet
Copy link

Just ran into this issue and had to restart the computer a few times before I saw my distro running under wsl -l -v, fwiw. I ran through the fsutil commands, as well as manually decompressing the package in %localappdata%\Packages\CanonicalGroup... I'm guessing the VHD is compressed until it's started, at which point the conversion works?

@davidasync
Copy link

OK, I managed to resolve it by myself after some intensive Googling by running those two commands:

fsutil behavior set disableencryption 1
fsutil behavior set disablecompression 1

and rebooting.

Now it just works™

Nonetheless I dont remember changing anything about NTFS behavior myself. Could it be that my laptop manufacturer (Dell) did it? Maybe this should be handled more gracefully by WSL2. Could WSL2 set those properties per virtual hard disk file to prevent others from hitting this?

Based on my experience, because of this command, I cannot install windows terminal from windows store, so I set it back to zero,
cheers!

@ghost
Copy link

ghost commented Aug 17, 2021

Decompressing the *.vhdx file won't work as you still won't be able to get the network connection from WSL2. Unless you don't plan to apt update ever again that is.

@lostsequence
Copy link

Okay, after some lucky guesses I've confirmed it and I have an addendum to the workaround (cc @caffme in case this works for you as well) - go to the Ubuntu directory in your profile folder (something like %USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents" (and I suppose "Encrypt contents" as well if that's checked). When it asks whether you want to apply to just this folder or to all subfolders and files, you can say "just this folder", because all you're doing is clearing that "compress" flag. After that the wsl --set-version command should work.

Worked for me, thanks

@TWiStErRob
Copy link

Please in the error message at least tell me which file is wrong!

Yes, I have compressed LocalState\ext4.vhdx to save space, that was 2 years ago, and until recently everything just worked fine. Now, 2 years later,

I get a random error when starting Ubuntu, and I get a flash of a command line when starting bash. Please add some user facing error handling, compressing and encrypting disks nowadays is pretty commonplace, isn't it?

@SayakMukhopadhyay
Copy link

I have been working on an issue which relates to this and so far I have not been able to reproduce this in windows 11. Does this issue occur when only the vhdx is compressed or the entire drive is compressed?

@TWiStErRob
Copy link

TWiStErRob commented Apr 18, 2022

Should be either. My drive was not compressed, only a select few space-wasting files. I'm still on Windows 10 ("the last" 😅) though.

@aniziki
Copy link

aniziki commented Jun 16, 2022

Dropping in to say I had this issue and I couldn't solve it for the life of me until I realized the drive was set for "Compress this drive to save disk space" in Disk Management
Unchecked box and formatted drive solved it for me
Hope it helps someone :)

@imoknu
Copy link

imoknu commented Jun 21, 2022

Ran into same error: 0xc03a0001a. Tried advice from @squeaky-pl
fsutil behavior set disableencryption 1
fsutil behavior set disablecompression 1
with restart

Then advice from @dmchurch to locate
%USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents"
with restart

Still encountered error. Then...tried checking the "compress contents" and applying the change. Received an error that compression couldn't be applied since the location was de-compressed (from executing the fsutil command previously). So unchecked the box and applied (to all subfolders). Did the same for the other folder in the Packages folder (there were two CanonicalGroupLimited Ubuntu folders), restarted and the install went through...finally!

Wanted to let folks know in case it was helpful to someone who tried this advice already and was still running into the issue.

Thanks @dmchurch and @squeaky-pl. After combining your advice and undoing/redoing the folder compression, it worked!

@alxd
Copy link

alxd commented Aug 10, 2022

Okay, after some lucky guesses I've confirmed it and I have an addendum to the workaround (cc @caffme in case this works for you as well) - go to the Ubuntu directory in your profile folder (something like %USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents" (and I suppose "Encrypt contents" as well if that's checked). When it asks whether you want to apply to just this folder or to all subfolders and files, you can say "just this folder", because all you're doing is clearing that "compress" flag. After that the wsl --set-version command should work.

this worked for me as well!

@daygumon
Copy link

daygumon commented Sep 7, 2022

Okay, after some lucky guesses I've confirmed it and I have an addendum to the workaround (cc @caffme in case this works for you as well) - go to the Ubuntu directory in your profile folder (something like %USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents" (and I suppose "Encrypt contents" as well if that's checked). When it asks whether you want to apply to just this folder or to all subfolders and files, you can say "just this folder", because all you're doing is clearing that "compress" flag. After that the wsl --set-version command should work.

Deselecting only the UbuntuonWindows's LocalState didn't solve it for me
C:\Users\%USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState
but also deselecting the Ubuntu's LocalState solved it.
C:\Users\%USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc\LocalState

I didn't even have to restart anything before running --set-version Ubuntu 2 again.

Thank you.

@xantari
Copy link

xantari commented Sep 14, 2022

@krzys-h is spot on, that KB4023057 appears to compress that folder %USERPROFILE%\AppData\Local\Packages\ but the KB article says it will uncompress it when it's done, but that is false. It keeps everything compressed all the time, and thus this super long thread on why a simple command line "wsl --set-version ubuntu 2" doesn't work.

@m4ttgit
Copy link

m4ttgit commented Oct 17, 2022

Okay, after some lucky guesses I've confirmed it and I have an addendum to the workaround (cc @caffme in case this works for you as well) - go to the Ubuntu directory in your profile folder (something like %USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents" (and I suppose "Encrypt contents" as well if that's checked). When it asks whether you want to apply to just this folder or to all subfolders and files, you can say "just this folder", because all you're doing is clearing that "compress" flag. After that the wsl --set-version command should work.

WORKS for me . Thanks @dmchurch

@grjkz
Copy link

grjkz commented Feb 16, 2023

Okay, after some lucky guesses I've confirmed it and I have an addendum to the workaround (cc @caffme in case this works for you as well) - go to the Ubuntu directory in your profile folder (something like %USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents" (and I suppose "Encrypt contents" as well if that's checked). When it asks whether you want to apply to just this folder or to all subfolders and files, you can say "just this folder", because all you're doing is clearing that "compress" flag. After that the wsl --set-version command should work.

Works for me as of today. Difference was that I uncompressed all subfolders from the distro folder but finally got a "Conversion complete" message when setting version to 2.

@IshanKumar22
Copy link

IshanKumar22 commented Mar 13, 2023

EDIT: I reinstalled kali and it is working.


I am running a Kali Linux distribution on D drive and am getting the same error even after unchecking compress option and applying it to every possible file and folder and subfolder.
image
File system:
image

@Sobanim
Copy link

Sobanim commented Nov 30, 2023

Okay, after some lucky guesses I've confirmed it and I have an addendum to the workaround (cc @caffme in case this works for you as well) - go to the Ubuntu directory in your profile folder (something like %USERPROFILE%\AppData\Local\Packages\CanonicalGroupLimited...), right click on "LocalState", Properties, Advanced, deselect "Compress contents" (and I suppose "Encrypt contents" as well if that's checked). When it asks whether you want to apply to just this folder or to all subfolders and files, you can say "just this folder", because all you're doing is clearing that "compress" flag. After that the wsl --set-version command should work.

For me it's now works 😠 I have a problem with docker and to larg space for wsl (36 Gb).

I tried to do something about it. I deleted all the docker containers, but only 3 GB freed up and they ran out after 2 weeks.
I was looking for other ways to get more free space. I found this method https://stackoverflow.com/questions/70946140/docker-desktop-wsl-ext4-vhdx-too-large
But I got the error: DiskPart has encountered an error: The specified file is encrypted and the user does not have the ability to decrypt it. See the System Event Log for more information.

I also came across this command that could help me: Optimize-VHD -Path 'D:\WpSystem\S-1-5-21-1303700208-3966646809-669386113-1001\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc\ LocalState\ext4.vhdx' -Mode Full
But this command writes an error: The specified file is encrypted and the user does not have the ability to decrypt it.

And when I try to remove encryption, I get this
2023-11-30_09-55
How can I decrypt wsl localState?

@nexus-chebykin
Copy link

This error might also occur if you have manually set your WSL instance into sparse mode.
Turns out there are two ways to do it: in .wslconfig and in terminal via wsl --manage Ubuntu-20.04 --set-sparse true.
To check whether your vhd is in sparse mode you should go to its LocalState folder and check if ext4.vhdx size on disk is smaller than its actual size.
To unsparse it you should remove sparseVhd=True from .wslconfig and do wsl --manage Ubuntu-20.04 --set-sparse false.

@erickkeven
Copy link

I deleted the entire CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc folder and my Ubuntu no longer works at all. I've tried everything, the error that appears in the Ubuntu application is Installing, this may take a few minutes...
WslRegisterDistribution failed with error: 0x80070002
Error: 0x80070002 The system cannot find the specified file.
and the other terminal error when I try to run a command such as: wsl
this other error message appears
The system cannot find the specified path.
Error code: Wsl/CallMsi/ERROR_PATH_NOT_FOUND

print1
print2
Does anyone know what to do?

@ForbiddenEra
Copy link

ForbiddenEra commented Mar 28, 2024

Wow; I did not know this was a thing.

So, my ext4.vhdx for Docker Desktop was a bit big and I figured I'd enable NTFS compression on it. Didn't save much anyway, but then I started having WSL issues after, hadn't used Docker Desktop in a few days but after rebooting after a Windows Update (which was the first thing I blamed, of course) WSL instances weren't starting up (non-Docker ones, I wasn't even using Docker at the time) and I was able to kill WSL in task manager and get my Ubuntu instance running. I couldn't even kill WSL using wsl --shutdown or by stopping it in services.msc or even PS Get-Service WSLService | Stop-Service (though I could stop/restart LxssManager with PS.

Didn't think about it for a few days and needed Docker today and it wouldn't start, just hung. Tried the usual, Windows Update, Reboot, restart services, nothing.

Checked Event Viewier:
image

Dunno if anything broke while it was compressing after enabling compression; 7-zip seems to read it file other than mentioning a header error.

Can't disable compression on the file either now:
image

Same error using compact.exe.. Figured I'd copy it to another drive and copy back to disable compression; just as I was typing this I got an error about not enough disk space, the target drive definitely has enough space so I was thinking 'WTF' again but now my C: is completely filled (I guess it decompressing it on the same drive before copying?), space isn't being reclaimed after the copy failure and I don't even know where to start looking to purge the cache.

I'm about to just obliterate it, it's 99.99% Docker cache and ephermal stuff, but I'd still rather not.

And after looking; as I suspected it's not even a visible file but now my C: is technically corrupt.

image
image

So, I delete a bunch of stuff to make sure I have well over the 150gb that the VHDX is free, try copying again. Got a successful copy to another drive, but again the journal didn't release the space after copy; now have 300gb stuck in the journal. Ugh. Now to delete the original, reboot, hope it releases, if not, kill/recreate the journal and cross fingers I don't have to image the drive, format, copy back with crossed fingers, or worse, re-install. I almost miss Win3.1/Win95/Win98 days when re-install was a daily/weekly thing, now I'm old and crusty and don't want to set everything up again :'(

Ugh; and of course, the file doesn't want to delete.

Well, after a reboot that I almost gave up waiting for, free space is as expected and the file was able to be deleted. Let's hope this hell is over.

image

Phew. Even got my Docker back after copying the VHDX back over.

Note to self and anyone else; Don't ever use NTFS compression on WSL VHDX files! Ever! EVER!@#$

@leodagostino
Copy link

leodagostino commented Apr 29, 2024

For me, nothing above was working, I just kept getting the "must not be encrypted or compressed" error. Even wsl --manage Ubuntu-20.04 --set-sparse false as mentioned by @nexus-chebykin didn't seem to work until I manually set sparseVhd=false in the .wsl_config file. Then, since I'm on Windows 10 home, proceeding with (in powershell):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk

@abalter
Copy link

abalter commented Jun 28, 2024

My situation was a little different than @leodagostino.

  • I had been using sparseVhd=true in my .wsl_config file.
  • I was getting the error about "encrypted or compressed"
  • I tried setting it to "false" and tried restarting WSL (wsl --shutdown then rebooting linux).
  • Was still getting the error
  • I ran wsl --manage Ubuntu --set-sparse false
  • When it completed, I was able to attach the vdisk without error

@RandallFlagg
Copy link

I solved this issue by runnig the following from here:
fsutil sparse setflag “C:\Users\Public\Documents\Hyper-V\Virtual hard disks\testserver.vhdx” 0

@sn-o-w
Copy link

sn-o-w commented Aug 11, 2024

The error "The requested operation could not be completed due to a virtual disk system limitation. Virtual hard disk files must be uncompressed and unencrypted and must not be sparse" could also appear if you are using Windows Subsystem for Android (WSA).

Using these commands should do the trick:

fsutil behavior set disableencryption 1
fsutil behavior set disablecompression 1

+

fsutil sparse setflag “%LOCALAPPDATA%\Packages\MicrosoftCorporationII.WindowsSubsystemForAndroid_8wekyb3d8bbwe\LocalCache\userdata.vhdx” 0
fsutil sparse setflag “%LOCALAPPDATA%\Packages\MicrosoftCorporationII.WindowsSubsystemForAndroid_8wekyb3d8bbwe\LocalCache\metadata.vhdx” 0

(We don't have any --set-sparse option for WSA.)

You might also need to manually untick "Compress contents to save disk space" (Properties -> Advanced) for userdata.vhdx and metadata.vhdx.

Then, if you want to shrink the VHDX files, you should be able to run via PowerShell:

Optimize-VHD -Path "$env:LOCALAPPDATA\Packages\MicrosoftCorporationII.WindowsSubsystemForAndroid_8wekyb3d8bbwe\LocalCache\*.vhdx" -Mode Full

Another alternative for Optimize-VHD is by using diskpart: #4103 (comment) (mount both VHDX files and compact them, one by one)

After you shrink the VHDX files, set sparse back:

fsutil sparse setflag “%LOCALAPPDATA%\Packages\MicrosoftCorporationII.WindowsSubsystemForAndroid_8wekyb3d8bbwe\LocalCache\userdata.vhdx” 1
fsutil sparse setflag “%LOCALAPPDATA%\Packages\MicrosoftCorporationII.WindowsSubsystemForAndroid_8wekyb3d8bbwe\LocalCache\metadata.vhdx” 1

And WSA will work afterward.

I leave this comment since this topic is very popular among Google search results, and it might be useful for people using WSA.

@puzzle9
Copy link

puzzle9 commented Sep 10, 2024

https://learn.microsoft.com/windows/wsl/wsl-config

C:\Users\[user]\.wslconfig

[experimental]
sparseVhd=false

@oboff
Copy link

oboff commented Sep 21, 2024

I solved this issue by runnig the following from here: fsutil sparse setflag “C:\Users\Public\Documents\Hyper-V\Virtual hard disks\testserver.vhdx” 0

Been searching for a while how to recover a .vhdx drive that was made Windows-unmountable by nbd (in Linux). Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug failure-to-launch failure to launch workaround-available wsl2 Issue/feature applies to WSL 2
Projects
None yet
Development

No branches or pull requests