cabal-testsuite failures on Windows due to the max path length #6271

phadej · 2019-10-01T13:47:08Z

I trying disable few tests on windows & appveyor, they weren't run previously anyway.

cabal-testsuite\PackageTests\Regression\T5309\cabal.test.hs
cabal-testsuite\PackageTests\Regression\T5677\cabal.test.hs
cabal-testsuite\PackageTests\Backpack\Includes3\cabal-internal.test.hs
cabal-testsuite\PackageTests\Backpack\Includes3\cabal-external.test.hs
cabal-testsuite\PackageTests\Backpack\Includes2\cabal-internal.test.hs
cabal-testsuite\PackageTests\Backpack\Includes2\cabal-external.test.hs
cabal-testsuite\PackageTests\Backpack\bkpcabal01\cabal.test.hs

This could solve some non-determinism in output of cabal-testsuite which we experienced on appveyor, i.e. #6271

phadej · 2019-11-28T04:54:53Z

Does

C:\WORK\cabal-testsuite\PackageTests\Backpack\Includes3\cabal-internal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\Includes3-0.1.0.0\l\indef\Includes3-0.1.0.0-inplace-indef+AZNSCvlbjiwGW3rUT5hCYh\build\Includes3-0.1.0.0-inplace-indef+AZNSCvlbjiwGW3rUT5hCYh\autogen\: openBinaryTempFileWithDefaultPermissions:

Look like path length limitation? Is there anything one can do to make it not fail?

Mistuke · 2019-11-28T14:47:02Z

Yeah that path is way too long. Which ghc and Cabal is this?

Mistuke · 2019-11-28T14:48:51Z

You can try this before the build/testsuite raaz-crypto/raaz@78a8459

For newer cabal and ghc you can install ghc-jailbreak which would take care of it too.

phadej · 2019-11-28T15:30:03Z

@Mistuke, Cabal is master. In cabal-testsuite, we indeed have quite long CWD to begin with. Making X: is a nice trick, have to try that.

But the GHC used, is 8.6.5, will ghc-jailbreak work with it, or does one need 8.8?

phadej · 2019-11-28T15:30:54Z

I'd try ghc-jailbreak if it works, before making X: trick. I recall you blogged about it, but I cannot find that anymore, do you have a link?

Mistuke · 2019-11-28T17:25:56Z

It depends. I'm not quite sure who is issuing this error.. If it's coming from cabal or ghc. openBinaryTempFileWithDefaultPermissions sounds like it's coming from base openBinaryTempFileWithDefaultPermissions in which case only the X: trick will work until ghc 8.12 because the temp file function in base doesn't use the MAX_PATH workaround and ghc-jailbreak only patches the tools ghc depends on and not ghc itself.

Blog post is at https://hub.zhox.com/posts/bypassing-max_path-limit-on-windows/ but I think it won't work in this case.

Mistuke · 2019-11-28T17:27:08Z

Alternative is to backport the temp file creation from 8.12 to cabal As a compat fix.

phadej · 2019-11-28T18:20:06Z

If openBinaryTempFileWithDefaultPermissions can be backported without FFI, then I'm open to that idea.

It's used in one place; writeFileAtomic :: FilePath -> BS.ByteString -> NoCallStackIO (),
so if that can be done somehow better on Win32, even better. (Now it's creating tmp file, and doing renameFile)

Mistuke · 2019-11-28T22:15:41Z

If openBinaryTempFileWithDefaultPermissions can be backported without FFI, then I'm open to that idea.

No it's mostly FFI. It gets complicated because openTemp and friends need to be atomic and work under threaded conditions. That means doing it in Haskell is tricky. Technically speaking at the moment cabal isn't threadsafe on Windows. Practically it probably is good enough because unlike hsc2hs and ghc you do a lot of work between invocations of openTemp.

So here are the options you have:

Shorten the paths using subst, no code change but not a "solution". As in, people will still hit it.
Backport openTemp and friends from GHC 8.10 and add some extra code to handle long paths, GCC won't like it so you'll need ghc-jailbreak as well untill GHC 8.10.
Backport openTemp and friends from GHC 8.12. Code is much simpler and handles long paths, but GCC won't like it until GHC 8.10 so you'll need ghc-jailbreak.
Change the temp path the function uses to something shorter.
XFAIL the test and re-enable it in two GHC releases or whenever the new I/O manager ships.

None of them very ideal solutions..

jneira · 2022-03-09T10:12:36Z

cabal-testsuite\PackageTests\Regression\T6906\cabal.test.hs also fails in windows for ghc-8.4.4 (see https://github.com/jneira/cabal/runs/5471916886?check_suite_focus=true#step:16:2319)

jneira · 2022-03-09T10:29:49Z

Trying to merge github workflows i noted lot of tests which works in ghc-8.6.5 with this invocation:

cabal v2-run cabal-testsuite:cabal-tests -- -j1 --with-cabal=dist-newstyle\build\x86_64-windows\ghc-${{ matrix.ghc }}\cabal-install-3.7.0.0\x\cabal\build\cabal\cabal.exe

does not work with this other generated by validate.sh:

D:\a\cabal\cabal\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-testsuite-3\build\cabal-tests\cabal-tests.exe --builddir=/d/a/cabal/cabal/dist-val-ghc-8.6.5/build/x86_64-windows/ghc-8.6.5/cabal-testsuite-3 --with-cabal=D:\a\cabal\cabal\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-install-3.7.0.0\x\cabal\build\cabal\cabal.exe -j2 --hide-successes

Enabling verbose in the succesful one i noted the test works with this log:

creating D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp

"C:\ProgramData\chocolatey\lib\ghc.8.6.5\tools\ghc-8.6.5\bin\ghc.exe" "--make" "-no-link" "-fbuilding-cabal-package" "-O" "-static" "-outputdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-odir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-hidir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-stubdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-i" "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-i." "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen" "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen" "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen" "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen" "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-optP-include" "-optPD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen\cabal_macros.h" "-hide-all-packages" "-Wmissing-home-modules" "-no-user-package-db" "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\home\.cabal\store\ghc-8.6.5\package.db" "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\packagedb\ghc-8.6.5" "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\package.conf.inplace" "-package-id" "base-4.12.0.0" "-package-id" "directory-1.3.3.0" "-XHaskell2010" ".\MyCustomPreprocessor.hs" "-hide-all-packages"

[1 of 1] Compiling Main             ( MyCustomPreprocessor.hs, D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp\Main.o )

The broken one failed in the same step with:

creating D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp

"C:\ProgramData\chocolatey\lib\ghc.8.6.5\tools\ghc-8.6.5\bin\ghc-8.6.5.exe" "--make" "-no-link" "-fbuilding-cabal-package" "-O" "-static" "-outputdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-odir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-hidir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-stubdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-i" "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-i." "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen" "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen" "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen" "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen" "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" "-optP-include" "-optPD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen\cabal_macros.h" "-hide-all-packages" "-Wmissing-home-modules" "-no-user-package-db" "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\home\.cabal\store\ghc-8.6.5\package.db" "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\packagedb\ghc-8.6.5" "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\package.conf.inplace" "-package-id" "base-4.12.0.0" "-package-id" "directory-1.3.3.0" "-XHaskell2010" ".\MyCustomPreprocessor.hs" "-hide-all-packages"

[1 of 1] Compiling Main             ( MyCustomPreprocessor.hs, D:\\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp\Main.o )

Assembler messages:

Fatal error: can't create D:\\\\a\\\\cabal\\\\cabal\\\\cabal-testsuite\\\\PackageTests\\\\CustomPreProcess\\\\cabal.dist\\\\work\\\\.\\\\dist\\\\build\\\\x86_64-windows\\\\ghc-8.6.5\\\\internal-preprocessor-test-0.1.0.0\\\\build\\\\my-custom-preprocessor\\\\my-custom-preprocessor-tmp\\Main.o: No such file or directory

`gcc.exe' failed in phase `Assembler'. (Exit code: 1)

The two ghc calls have exactly the same paths. The only difference is the bad one uses ghc-8.6.5.exe and the good one ghc.exe but i cant believe that could be the culprit.

Observe the double dash in the broken one maybe related?

jneira · 2022-03-10T20:25:59Z

Cant reproduce locally with

"D:\dev\ws\haskell\cabal\.\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-testsuite-3\build\cabal-tests\cabal-tests.exe"  --builddir="D:\dev\ws\haskell\cabal\.\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-testsuite-3" ".\cabal-testsuite\PackageTests\CustomPreProcess\setup.test.hs" --with-cabal="D:\dev\ws\haskell\cabal\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-install-3.7.0.0\x\cabal\build\cabal\cabal.exe"

🤦

jneira · 2022-03-11T08:49:33Z

I am tempted to mark those tests as broken for ghc >= 8.6.5 as we are gonna test 8.10, 9.0 and hopefully 9.2 in windows but i would prefer not having to do.
@Mistuke know you are busy but any kind of help would be much appreciated

Mistuke · 2022-03-11T11:29:56Z

The Paths aren't exactly the same:

The working one has

[1 of 1] Compiling Main             ( MyCustomPreprocessor.hs, D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp\Main.o )

and the broken one

[1 of 1] Compiling Main             ( MyCustomPreprocessor.hs, D:\\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp\Main.o )

Notice the difference D:\ vs D:\\. Cabal double escapes these. So the first path becomes:

> "D:\\a\\\\cabal\\\\cabal\\\\cabal-testsuite\\\\PackageTests\\\\CustomPreProcess\\\\cabal.dist\\\\work\\\\.\\\\dist\\\\build\\\\x86_64-windows\\\\ghc-8.6.5\\\\internal-preprocessor-test-0.1.0.0\\\\build\\\\my-custom-preprocessor\\\\my-custom-preprocessor-tmp\\Main.o".length
264

Which is just below the limit... and the broken

> "D:\\\\a\\\\cabal\\\\cabal\\\\cabal-testsuite\\\\PackageTests\\\\CustomPreProcess\\\\cabal.dist\\\\work\\\\.\\\\dist\\\\build\\\\x86_64-windows\\\\ghc-8.6.5\\\\internal-preprocessor-test-0.1.0.0\\\\build\\\\my-custom-preprocessor\\\\my-custom-preprocessor-tmp\\Main.o".length
266

which is above the limit. Something has done some weird double escaping here. don't quite understand the escaping logic.. It doesn't look consistent.

We only started patching the toolchain for long paths in GHC 8.10. With GHC 8.6 we patched GHC itself, but binutils is still bound.

For this particular test, if you can fix the over-escaping it would bring you back under.

D:/a/cabal/cabal/cabal-testsuite/PackageTests/CustomPreProcess/cabal.dist/work/./dist/build/x86_64-windows/ghc-8.6.5/internal-preprocessor-test-0.1.0.0/build/my-custom-preprocessor/my-custom-preprocessor-tmp/Main.o

is equivalent but much shorter at 214 chars.

jneira · 2022-03-11T12:37:42Z

many thanks @Mistuke, i had observed the double quoting behaviour, but i dont understand why invoking the cabal test suite with cabal run cabal-testsuite -- --test-args or invoking directly the executable under build dir with $(cabal-plan list-bin cabal-testsuite) --test-args could make that change

The full invocation of the working one is:

cabal v2-run cabal-testsuite:cabal-tests -- -j1 -v --with-cabal=dist-newstyle\build\x86_64-windows\ghc-8.6.5\cabal-install-3.7.0.0\x\cabal\build\cabal\cabal.exe

... without building the executable before that

The full invocation of the not working one is:

D:\a\cabal\cabal\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-testsuite-3\build\cabal-tests\cabal-tests.exe --builddir=D:\a\cabal\cabal\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-testsuite-3 --with-cabal=D:\a\cabal\cabal\dist-val-ghc-8.6.5\build\x86_64-windows\ghc-8.6.5\cabal-install-3.7.0.0\x\cabal\build\cabal\cabal.exe -j2 --hide-successes

... building the executable with cabal build in a msys2 bash console

So one hypothesis could be cabal run does some magic to avoid double escape.

I also tried to shorthen the dist-val-ghc-8.6.5 build dir to dist-val8.6.5 with no luck just in case.

Dont understand neither why cant reproduce it locally.

I tried the not working one in ci under msys2 bash, raw cmd and powershell and it did not work

jneira · 2022-03-11T12:50:11Z

which is above the limit. Something has done some weird double escaping here. don't quite understand the escaping logic.. It doesn't look consistent.

and the bad ghc call does not have any double slash in the ghc invocation, so it seems ghc is doing it depending on some stateful thing:

"C:\ProgramData\chocolatey\lib\ghc.8.6.5\tools\ghc-8.6.5\bin\ghc-8.6.5.exe"
  "--make" "-no-link" "-fbuilding-cabal-package" "-O" "-static" 
  "-outputdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" 
  "-odir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" 
  "-hidir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" 
  "-stubdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp" 
  "-i" "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-i." "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen"
  "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen"
  "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen"
  "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen"
  "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-optP-include"
  "-optPD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen\cabal_macros.h"
  "-hide-all-packages" "-Wmissing-home-modules" "-no-user-package-db"
  "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\home\.cabal\store\ghc-8.6.5\package.db"
  "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\packagedb\ghc-8.6.5"
  "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\package.conf.inplace"
  "-package-id" "base-4.12.0.0" "-package-id" "directory-1.3.3.0" "-XHaskell2010" ".\MyCustomPreprocessor.hs" "-hide-all-packages"

good one, identical but for the ghc.exe instead ghc-8.6.5.exe, i also tried desperately use ghc.exein the bas one with no luck

"C:\ProgramData\chocolatey\lib\ghc.8.6.5\tools\ghc-8.6.5\bin\ghc.exe"
  "--make" "-no-link" "-fbuilding-cabal-package" "-O" "-static"
  "-outputdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-odir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-hidir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-stubdir" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-i" "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-i." "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen"
  "-iD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen"
  "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen"
  "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\global-autogen"
  "-ID:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\my-custom-preprocessor-tmp"
  "-optP-include"
  "-optPD:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\build\my-custom-preprocessor\autogen\cabal_macros.h"
  "-hide-all-packages" "-Wmissing-home-modules" "-no-user-package-db"
  "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\home\.cabal\store\ghc-8.6.5\package.db"
  "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\packagedb\ghc-8.6.5"
  "-package-db" "D:\a\cabal\cabal\cabal-testsuite\PackageTests\CustomPreProcess\cabal.dist\work\.\dist\build\x86_64-windows\ghc-8.6.5\internal-preprocessor-test-0.1.0.0\package.conf.inplace"
  "-package-id" "base-4.12.0.0" "-package-id" "directory-1.3.3.0" "-XHaskell2010" ".\MyCustomPreprocessor.hs" "-hide-all-packages"

jneira · 2022-03-11T12:53:00Z

well maybe the output of the call is the same but the double slashes are removed printing out the log 🤔

jneira · 2022-03-11T13:00:24Z

Mistuke · 2022-03-11T13:13:46Z

and the bad ghc call does not have any double slash in the ghc invocation, so it seems ghc is doing it depending on some stateful thing:

Ah yes, prior to GHC 9, the versioned files used to be wrappers, and from 9, it's the unversioned one.

those wrappers are ancient and re-escape things. See https://github.com/ghc/ghc/blob/ghc-8.6.5-release/driver/utils/cwrapper.c
That's the second level of escaping

looks like they don't escape /. That code needs to be smarted, I rewrote it at some point but never submitted it upstream.. You can try normalizing the paths to / instead of \ when calling GHC.

jneira · 2022-03-11T14:17:14Z

Ah yes, prior to GHC 9, the versioned files used to be wrappers, and from 9, it's the unversioned one.

those wrappers are ancient and re-escape things. See https://github.com/ghc/ghc/blob/ghc-8.6.5-release/driver/utils/cwrapper.c That's the second level of escaping

wow, million thanks for that, I would swear I tried it with the main ghc.exe but trying it again, that should fix the calls too, no?

Mistuke · 2022-03-12T20:19:22Z

wow, million thanks for that, I would swear I tried it with the main ghc.exe but trying it again, that should fix the calls too, no?

It should, for 9.x GHC HQ flipped which one is the main though. though 9.x shouldn't have a problem with long paths.

Linking haskell#6271 (comment)

phadej added priority: high 🔥 type: bug cabal-install: Backpack cabal-testsuite Cabal: Backpack labels Oct 1, 2019

phadej mentioned this issue Oct 1, 2019

UNEXPECTED FAIL: cabal-testsuite\PackageTests\CustomPreProcess\setup.test.hs Command exited with code 1 #6269

Closed

phadej added a commit that referenced this issue Nov 28, 2019

fetchRepoTarball output is non-deterministic

3ad0e47

This could solve some non-determinism in output of cabal-testsuite which we experienced on appveyor, i.e. #6271

phadej mentioned this issue Nov 28, 2019

fetchRepoTarball output is non-deterministic #6385

Merged

phadej added a commit that referenced this issue Nov 28, 2019

fetchRepoTarball output is non-deterministic

32099f9

This could solve some non-determinism in output of cabal-testsuite which we experienced on appveyor, i.e. #6271

phadej mentioned this issue Dec 25, 2019

T5677 rename prog to program, case-sensitive type-systems break #6463

Merged

jneira added type: testing Issues about project test suites and removed priority: high 🔥 labels Mar 9, 2022

jneira changed the title ~~cabal-testsuite failures on Windows~~ cabal-testsuite failures on Windows due to the max path length Mar 9, 2022

jneira mentioned this issue Mar 9, 2022

Merge workflows jneira/cabal#5

Closed

3 tasks

jneira mentioned this issue Mar 14, 2022

Merge workflows #8045

Merged

jneira added a commit to jneira/cabal that referenced this issue Mar 15, 2022

Add comment about -w ghc workaround

22e33d7

Linking haskell#6271 (comment)

jasagredo added the platform: windows label Jul 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cabal-testsuite failures on Windows due to the max path length #6271

cabal-testsuite failures on Windows due to the max path length #6271

phadej commented Oct 1, 2019 •

edited

Loading

phadej commented Nov 28, 2019

Mistuke commented Nov 28, 2019

Mistuke commented Nov 28, 2019

phadej commented Nov 28, 2019

phadej commented Nov 28, 2019

Mistuke commented Nov 28, 2019

Mistuke commented Nov 28, 2019

phadej commented Nov 28, 2019 •

edited

Loading

Mistuke commented Nov 28, 2019

jneira commented Mar 9, 2022 •

edited

Loading

jneira commented Mar 9, 2022 •

edited

Loading

jneira commented Mar 10, 2022 •

edited

Loading

jneira commented Mar 11, 2022

Mistuke commented Mar 11, 2022

jneira commented Mar 11, 2022

jneira commented Mar 11, 2022 •

edited

Loading

jneira commented Mar 11, 2022

jneira commented Mar 11, 2022

Mistuke commented Mar 11, 2022

jneira commented Mar 11, 2022

Mistuke commented Mar 12, 2022

cabal-testsuite failures on Windows due to the max path length #6271

cabal-testsuite failures on Windows due to the max path length #6271

Comments

phadej commented Oct 1, 2019 • edited Loading

phadej commented Nov 28, 2019

Mistuke commented Nov 28, 2019

Mistuke commented Nov 28, 2019

phadej commented Nov 28, 2019

phadej commented Nov 28, 2019

Mistuke commented Nov 28, 2019

Mistuke commented Nov 28, 2019

phadej commented Nov 28, 2019 • edited Loading

Mistuke commented Nov 28, 2019

jneira commented Mar 9, 2022 • edited Loading

jneira commented Mar 9, 2022 • edited Loading

jneira commented Mar 10, 2022 • edited Loading

jneira commented Mar 11, 2022

Mistuke commented Mar 11, 2022

jneira commented Mar 11, 2022

jneira commented Mar 11, 2022 • edited Loading

jneira commented Mar 11, 2022

jneira commented Mar 11, 2022

Mistuke commented Mar 11, 2022

jneira commented Mar 11, 2022

Mistuke commented Mar 12, 2022

phadej commented Oct 1, 2019 •

edited

Loading

phadej commented Nov 28, 2019 •

edited

Loading

jneira commented Mar 9, 2022 •

edited

Loading

jneira commented Mar 9, 2022 •

edited

Loading

jneira commented Mar 10, 2022 •

edited

Loading

jneira commented Mar 11, 2022 •

edited

Loading