Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

appveyor mingw64 build fails #6248

Closed
belegdol opened this issue Feb 1, 2020 · 10 comments
Closed

appveyor mingw64 build fails #6248

belegdol opened this issue Feb 1, 2020 · 10 comments

Comments

@belegdol
Copy link
Contributor

belegdol commented Feb 1, 2020

Appveyor MINGW build has started failing 9 days ago:
https://ci.appveyor.com/project/startaq/mame/builds/30306506
I was able to reproduce it locally when using bash shell:

$ LANG=C make VERBOSE=1 SUBTARGET=tiny TOOLS=1 OPTIMIZE=2 -j12
GCC 9.2.0 detected
make -R verbose=1 -C build/projects/windows/mametiny/gmake-mingw64-gcc config=release64 WINDRES=/mingw64/bin/windres precompile
make[1]: Entering directory '/c/Users/beleg/source/repos/belegdol/mame/build/projects/windows/mametiny/gmake-mingw64-gcc'
make[1]: Leaving directory '/c/Users/beleg/source/repos/belegdol/mame/build/projects/windows/mametiny/gmake-mingw64-gcc'
make -R verbose=1 -C build/projects/windows/mametiny/gmake-mingw64-gcc config=release64 WINDRES=/mingw64/bin/windres
make[1]: Entering directory '/c/Users/beleg/source/repos/belegdol/mame/build/projects/windows/mametiny/gmake-mingw64-gcc'
Archiving libemu.a...
rm -f "../../../../mingw-gcc/bin/x64/Release/libemu.a"
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ar.exe: ../../../../mingw-gcc/bin/x64/Release/libemu.a../../../../mingw-gcc/obj/x64/Release/src/emu/addrmap.o: No such file or directory
make[2]: *** [emu.make:932: ../../../../mingw-gcc/bin/x64/Release/libemu.a] Error 1
make[1]: *** [Makefile:88: emu] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/c/Users/beleg/source/repos/belegdol/mame/build/projects/windows/mametiny/gmake-mingw64-gcc'
make: *** [makefile:1076: windows_x64] Error 2

Strangely enough build works in cmd shell started with win32env.bat.
It appears to me that there is a space missing between libemu.a and the path leading to addrmap.o. Any ideas what might have swallowed it? No commit between be36db0 and eb33990 touches anything global so I am guessing pacman update is to blame.

@cuavas
Copy link
Member

cuavas commented Feb 1, 2020

@Lord-Nightmare seemed to have a similar issue that he solved by rolling back the MinGW GNU make package in his MSYS2 environment.

@belegdol
Copy link
Contributor Author

belegdol commented Feb 1, 2020

Thanks for the tip, downgrading make from 4.3-1 to 4.2.1-1 seems to have done the trick. I will file a bug with msys2.

@Lord-Nightmare
Copy link
Contributor

Lord-Nightmare commented Feb 2, 2020

The error I ran into was "Archiving libformats.a...
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ar.exe: ../../../../mingw-gcc/bin/x64/Release/mame_arcade/libformats.a../../../../mingw-gcc/obj/x64/Release/src/lib/formats/2d_dsk.o: No such file or directory" and similarly the issue seems to be a dropped space between command line parameters.

Hypothesis: I'm guessing it is path length dependent; my mame compile base directory is in "c:\root\mame\mame" so depending on the exact length of the commands plus path (maybe at a 256 byte boundary?) a space is getting dropped, but only for make commands which happen to have the exact specific length/offset-of-space-characters to trigger it? I'm not sure this is the reason, but it could be the cause... It needs further testing, which I currently can't do as I'm going to be upgrading to a new system within the next few days.

@belegdol
Copy link
Contributor Author

belegdol commented Feb 3, 2020

Msys2 developer has referred to the make release notes:

WARNING: Backward-incompatibility! Previously appending using '+=' to an empty variable would result in a value starting with a space. Now the initial space is only added if the variable already contains some value. Similarly, appending an empty string does not add a trailing space.

@asavah
Copy link

asavah commented Feb 3, 2020

A quick hack that fixes the build with make 4.3,
TARGET=mame build of mame0218 on linux succeeded with this

diff --git a/3rdparty/genie/src/host/scripts.c b/3rdparty/genie/src/host/scripts.c
index bbdebc0ef7..a5c13b461e 100644
--- a/3rdparty/genie/src/host/scripts.c
+++ b/3rdparty/genie/src/host/scripts.c
@@ -229,7 +229,7 @@ const char* builtin_scripts[] = {

        /* actions/make/make_cpp.lua */
        "-- --\npremake.make.cpp = { }\npremake.make.override = { }\npremake.make.makefile_ignore = false\nlocal cpp = premake.make.cpp\nlocal make = premake.make\nfunction premake.make_cpp(prj)\nlocal cc = premake.gettool(prj)\nlocal platforms = premake.filterplatforms(prj.solution, cc.platforms, \"Native\")\nlocal action = premake.action.current()\npremake.gmake_cpp_header(prj, cc, platforms)\npremake.gmake_cpp_configs(prj, cc, platforms)\ntable.sort(prj.allfiles)\nlocal objdirs = {}\nlocal additionalobjdirs = {}\nfor _, file in ipairs(prj.allfiles) do\nif path.issourcefile(file) then\nobjdirs[_MAKE.esc(path.getdirectory(path.trimdots(file)))] = 1\nend\nend\nfor _, custombuildtask in ipairs(prj.custombuildtask or {}) do\nfor _, buildtask in ipairs(custombuildtask or {}) do\nadditionalobjdirs[_MAKE.esc(path.getdirectory(path.getrelative(prj.location,buildtask[2])))] = 1\nend\nend\n_p('OBJDIRS := \\\\')\n_p('\\t$(OBJDIR) \\\\')\nfor dir, _ in iter.sortByKeys(objdirs) do\n_p('\\t$(OBJDIR)/%s \\\\', dir)\nend\nfor dir, _"
-       " in iter.sortByKeys(additionalobjdirs) do\n_p('\\t%s \\\\', dir)\nend\n_p('')\n_p('RESOURCES := \\\\')\nfor _, file in ipairs(prj.allfiles) do\nif path.isresourcefile(file) then\n_p('\\t$(OBJDIR)/%s.res \\\\', _MAKE.esc(path.getbasename(file)))\nend\nend\n_p('')\n_p('.PHONY: clean prebuild prelink')\n_p('')\nif os.is(\"MacOSX\") and prj.kind == \"WindowedApp\" and not prj.options.SkipBundling then\n_p('all: $(OBJDIRS) $(TARGETDIR) prebuild prelink $(TARGET) $(dir $(TARGETDIR))PkgInfo $(dir $(TARGETDIR))Info.plist')\nelse\n_p('all: $(OBJDIRS) $(TARGETDIR) prebuild prelink $(TARGET)')\nend\n_p('\\t@:')\n_p('')\nif (prj.kind == \"StaticLib\" and prj.options.ArchiveSplit) then\n_p('define max_args')\n_p('\\t$(eval _args:=)')\n_p('\\t$(foreach obj,$3,$(eval _args+=$(obj))$(if $(word $2,$(_args)),$1$(_args)$(EOL)$(eval _args:=)))')\n_p('\\t$(if $(_args),$1$(_args))')\n_p('endef')\n_p('')\n_p('define EOL')\n_p('')\n_p('')\n_p('endef')\n_p('')\nend\n_p('$(TARGET): $(GCH) $(OBJECTS) $(LIBDEPS) $(EXTERNAL_LIBS) $(RESOUR"
+       " in iter.sortByKeys(additionalobjdirs) do\n_p('\\t%s \\\\', dir)\nend\n_p('')\n_p('RESOURCES := \\\\')\nfor _, file in ipairs(prj.allfiles) do\nif path.isresourcefile(file) then\n_p('\\t$(OBJDIR)/%s.res \\\\', _MAKE.esc(path.getbasename(file)))\nend\nend\n_p('')\n_p('.PHONY: clean prebuild prelink')\n_p('')\nif os.is(\"MacOSX\") and prj.kind == \"WindowedApp\" and not prj.options.SkipBundling then\n_p('all: $(OBJDIRS) $(TARGETDIR) prebuild prelink $(TARGET) $(dir $(TARGETDIR))PkgInfo $(dir $(TARGETDIR))Info.plist')\nelse\n_p('all: $(OBJDIRS) $(TARGETDIR) prebuild prelink $(TARGET)')\nend\n_p('\\t@:')\n_p('')\nif (prj.kind == \"StaticLib\" and prj.options.ArchiveSplit) then\n_p('define max_args')\n_p('\\t$(eval _args:=)')\n_p('\\t$(foreach obj,$3,$(eval _args+=$(obj))$(if $(word $2,$(_args)),$1 $(_args)$(EOL)$(eval _args:=)))')\n_p('\\t$(if $(_args),$1 $(_args))')\n_p('endef')\n_p('')\n_p('define EOL')\n_p('')\n_p('')\n_p('endef')\n_p('')\nend\n_p('$(TARGET): $(GCH) $(OBJECTS) $(LIBDEPS) $(EXTERNAL_LIBS) $(RESOUR"
        "CES) $(OBJRESP) $(LDRESP) | $(TARGETDIR) $(OBJDIRS)')\nif prj.kind == \"StaticLib\" then\nif prj.msgarchiving then\n_p('\\t@echo ' .. prj.msgarchiving)\nelse\n_p('\\t@echo Archiving %s', prj.name)\nend\nif (not prj.archivesplit_size) then\nprj.archivesplit_size=200\nend\nif (not prj.options.ArchiveSplit) then\n_p('ifeq (posix,$(SHELLTYPE))')\n_p('\\t$(SILENT) rm -f  $(TARGET)')\n_p('else')\n_p('\\t$(SILENT) if exist $(subst /,\\\\\\\\,$(TARGET)) del $(subst /,\\\\\\\\,$(TARGET))')\n_p('endif')\n_p('\\t$(SILENT) $(LINKCMD) $(LINKOBJS)' .. (os.is(\"MacOSX\") and \" 2>&1 > /dev/null | sed -e '/.o) has no symbols$$/d'\" or \"\"))\nelse\n_p('\\t$(call RM,$(TARGET))')\n_p('\\t@$(call max_args,$(LINKCMD),'.. prj.archivesplit_size ..',$(LINKOBJS))' .. (os.is(\"MacOSX\") and \" 2>&1 > /dev/null | sed -e '/.o) has no symbols$$/d'\" or \"\"))\n_p('\\t$(SILENT) $(LINKCMD_NDX)')\nend\nelse\nif prj.msglinking then\n_p('\\t@echo ' .. prj.msglinking)\nelse\n_p('\\t@echo Linking %s', prj.name)\nend\n_p('\\t$(SILENT) $(LINKCMD)"
        "')\nend\n_p('\\t$(POSTBUILDCMDS)')\n_p('')\n_p('$(TARGETDIR):')\npremake.make_mkdirrule(\"$(TARGETDIR)\")\n_p('$(OBJDIRS):')\nif (not prj.solution.messageskip) or (not table.contains(prj.solution.messageskip, \"SkipCreatingMessage\")) then\n_p('\\t@echo Creating $(@)')\nend\n_p('\\t-$(call MKDIR,$@)')\n_p('')\nif os.is(\"MacOSX\") and prj.kind == \"WindowedApp\" and not prj.options.SkipBundling then\n_p('$(dir $(TARGETDIR))PkgInfo:')\n_p('$(dir $(TARGETDIR))Info.plist:')\n_p('')\nend\n_p('clean:')\nif (not prj.solution.messageskip) or (not table.contains(prj.solution.messageskip, \"SkipCleaningMessage\")) then\n_p('\\t@echo Cleaning %s', prj.name)\nend\n_p('ifeq (posix,$(SHELLTYPE))')\n_p('\\t$(SILENT) rm -f  $(TARGET)')\n_p('\\t$(SILENT) rm -rf $(OBJDIR)')\n_p('else')\n_p('\\t$(SILENT) if exist $(subst /,\\\\\\\\,$(TARGET)) del $(subst /,\\\\\\\\,$(TARGET))')\n_p('\\t$(SILENT) if exist $(subst /,\\\\\\\\,$(OBJDIR)) rmdir /s /q $(subst /,\\\\\\\\,$(OBJDIR))')\n_p('endif')\n_p('')\n_p('prebuild:')\n_p('\\t$(PRE"
        "BUILDCMDS)')\n_p('')\n_p('prelink:')\n_p('\\t$(PRELINKCMDS)')\n_p('')\ncpp.pchrules(prj)\ncpp.fileRules(prj, cc)\ncpp.dependencyRules(prj)\nfor _, custombuildtask in ipairs(prj.custombuildtask or {}) do\nfor _, buildtask in ipairs(custombuildtask or {}) do\nlocal deps =  string.format(\"%s \",path.getrelative(prj.location,buildtask[1]))\nfor _, depdata in ipairs(buildtask[3] or {}) do\ndeps = deps .. string.format(\"%s \",path.getrelative(prj.location,depdata))\nend\n_p('%s: %s | $(TARGETDIR) $(OBJDIRS)'\n,path.getrelative(prj.location,buildtask[2])\n, deps\n)\nfor _, cmdline in ipairs(buildtask[4] or {}) do\nlocal cmd = cmdline\nlocal num = 1\nfor _, depdata in ipairs(buildtask[3] or {}) do\ncmd = string.gsub(cmd,\"%$%(\" .. num ..\"%)\", string.format(\"%s \",path.getrelative(prj.location,depdata)))\nnum = num + 1\nend\ncmd = string.gsub(cmd, \"%$%(<%)\", \"$<\")\ncmd = string.gsub(cmd, \"%$%(@%)\", \"$@\")\n_p('\\t$(SILENT) %s',cmd)\nend\n_p('')\nend\nend\n_p('-include $(OBJECTS:%%.o=%%.d)')\n_p('ifneq (,$("

@belegdol
Copy link
Contributor Author

belegdol commented Feb 3, 2020

Thanks, your fix worked! The non-hacky way is to edit 3rdparty/genie/src/actions/make/make_cpp.lua and then run

$ genie embed

Would you like to submit the PR upstream, or should I?

@belegdol
Copy link
Contributor Author

belegdol commented Feb 3, 2020

Here is the make_cpp.lua patch:

diff --git a/3rdparty/genie/src/actions/make/make_cpp.lua b/3rdparty/genie/src/actions/make/make_cpp.lua
index a4e92521bd..880d9d35bf 100644
--- a/3rdparty/genie/src/actions/make/make_cpp.lua
+++ b/3rdparty/genie/src/actions/make/make_cpp.lua
@@ -73,8 +73,8 @@
                if (prj.kind == "StaticLib" and prj.options.ArchiveSplit) then
                        _p('define max_args')
                        _p('\t$(eval _args:=)')
-                       _p('\t$(foreach obj,$3,$(eval _args+=$(obj))$(if $(word $2,$(_args)),$1$(_args)$(EOL)$(eval _args:=)))')
-                       _p('\t$(if $(_args),$1$(_args))')
+                       _p('\t$(foreach obj,$3,$(eval _args+=$(obj))$(if $(word $2,$(_args)),$1 $(_args)$(EOL)$(eval _args:=)))')
+                       _p('\t$(if $(_args),$1 $(_args))')
                        _p('endef')
                        _p('')
                        _p('define EOL')

@asavah
Copy link

asavah commented Feb 3, 2020

If you are familiar with genie - please submit the PR, I know absolutely nothing about genie.
I found the workaround by looking at the genearted makefiles and grepping for max_args.
I tried patching make_cpp.lua first and it obviously didn't work and then I patched scripts.c.

@smf-
Copy link
Member

smf- commented Feb 3, 2020 via email

@belegdol
Copy link
Contributor Author

belegdol commented Feb 4, 2020

I just tested with win32env.bat and it worked. win32env.bat appears to be using make-4.1, which also explains why it was not affected by the original issue.
The fix also works with make downgraded back to 4.2.1 with bash shell.

@belegdol belegdol closed this as completed Feb 6, 2020
AluisioASG added a commit to AluisioASG/nixpkgs that referenced this issue Oct 28, 2020
Builds currently fail with `ar` trying to operate on what are clearly
two paths concatenated together.  It stems from a backward-incompatible
change in Make:

> Previously appending using '+=' to an empty variable would result in
> a value starting with a space.  Now the initial space is only added
> if the variable already contains some value.  Similarly, appending an
> empty string does not add a trailing space.

This issue was first reported on the MAME repository proper
(mamedev/mame#6248), and affects libretro's
2016 snapshot as well.  A fix that is reported to work with previous
versions of Make was upstreamed to:
- GENie, the build system: bkaradzic/GENie#493
- MAME: mamedev/mame#6262
- libretro: libretro/mame2016-libretro#47

The fetched patch comes from the last of these.
AluisioASG added a commit to AluisioASG/nixpkgs that referenced this issue Oct 30, 2020
Builds currently fail with `ar` trying to operate on what are clearly
two paths concatenated together.  It stems from a backward-incompatible
change in Make:

> Previously appending using '+=' to an empty variable would result in
> a value starting with a space.  Now the initial space is only added
> if the variable already contains some value.  Similarly, appending an
> empty string does not add a trailing space.

This issue was first reported on the MAME repository proper
(mamedev/mame#6248), and affects libretro's
2016 snapshot as well.  A fix that is reported to work with previous
versions of Make was upstreamed to:
- GENie, the build system: bkaradzic/GENie#493
- MAME: mamedev/mame#6262
- libretro: libretro/mame2016-libretro#47

The fetched patch comes from the last of these.

(cherry picked from commit 8880179)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants