mingw builds

[real-or-random]

#922 got me interested in mingw, so I started playing around and reading. Here are some notes that may save others time and should be converted into a PR at some point. I'd appreciate any comments from people more familiar with the topic (@gmaxwell, @theuni).

secp256k1 has been written to support mingw builds. As demonstrated in #922, these builds (still) work in general. But a few caveats apply.

Here are some simple issues to start with:

• The Makefile is not written with Windows in mind. The most obvious issue is that they ignore the .exe extension, e.g., in make clean and for gen_context. Automake has explicit support for this, which we can rely easily rely on.
• When cross-compiling on Linux, it may happen that configure believes that we're not cross-compiling because it's possible to execute windows binaries with an installed Wine and binfmt_misc enables. (This happens on my system but not on CI). See https://stackoverflow.com/questions/13150631/cross-compiling-why-checking-whether-we-are-cross-compiling-no#comment40154987_13150651. This by itself is not an issue: There's per se nothing wrong with configure building gen_context.exe and using wine to execute it. It's a little bit cumbersome though and together with the previous item that means that the gen_context invocation will fail. (Replace gen_context.c with a Python implementation #919 may improve this but could create other problems because invoking Python portably could be difficult.)

The real fun starts with DLLs (shared libaries) which are a pain in the ass. When creating a DLL, one needs to declare exported symbols explictly using __declspec (dllexport). We currently do this via in secp256k1.h via SECP256K1_API when SECP256K1_BUILD is defined.

When using the library (i.e., SECP256K1_BUILD is not defined), one can declare the symbols explicitly using __declspec (dllimport). We currently don't do this because the only problem is a small overhead when calling a DLL function but the code still works. The reason to avoid __declspec (dllimport) is that static linking will fail. (I verified this by adding the declspec and configuring with --disable-shared.) Unfortunately there's no way of telling whether we'll link statically or not, so the best possible thing is to have a macro that the user can set. (See also https://www.sourceware.org/autobook/autobook/autobook_137.html, https://autotools.io/libtool/windows.html, #314).

• We currently don't have this macro. We could add it, maybe it's useful for someone and it won't create problems because it's not set by default.
• In order to be able to create DLLs at all, we need to add -no-undefined to the appropriate libtool LDFLAGS like libsecp256k1_la_LDFLAGS = -no-undefined. Otherwise libtool will tell us libtool: warning: undefined symbols not allowed in x86_64-w64-mingw32 shared libraries; building static only as seen on CI in Add mingw32-w64/wine CI build #922. A workaround for now and for testing is to call make with make LDFLAGS=-no-undefined. Apparently this flag does not hurt on Linux, and none of the guides I found mentions that this should be only set on Windows, so we can just set this unconditionally.
• When building the benchmarks, we'll still get warnings like ./.libs/lt-bench_internal.c:41:5: warning: '_putenv' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes] but these are in the C code of the libtool wrapper binary, not in our code. (On Windows, this is an entire wrapper binary and not just a sh script as on Linux). I have no idea if this is a libtool bug or the issue is on our side but apparently the wrapper binary works, so we can simply ignore these for now. Having warnings is not elegant though, so it may still be interesting to ask on the libtool list to see what's going on.
• According to the libtool and gnulib docs, we should add win32-dll to LT_INIT. For me and CI, it seems to work without this, but I guess we should simply add it.

Uff.

[real-or-random]

In order to be able to create DLLs at all, we need to add -no-undefined to the appropriate libtool LDFLAGS like libsecp256k1_la_LDFLAGS = -no-undefined

So, I think that forces the linker to error out if there are undefined symbols. That sounds good, independently of the mingw build process forcing you to set it. Unless I'm confused?

That's also my understanding. There may be cases in which -no-undefined is undesired, e.g., cyclic dependencies in libraries. But in our case all symbols should always be defined.

We currently don't have this macro. We could add it, maybe it's useful for someone and it won't create problems because it's not set by default.

It's pretty common for there to be some STATIC_BUILD-ish macro, I think that would be fine, but it's worth keeping in mind that except for FOSS developers cross-compile targeting windows, windows developers 95% of the time won't use any build system you provide (and 1% of the time will but will mangle it horribly). Even providing a vcproj doesn't work that well because inevitably the versions will mismatch what they want to use and they are a PITA to keep current. :P Best solution for windows users is to try to make things work with as little build intelligence as possible. Fortunately the library has been going in that direction.

Indeed. Independently of the changes proposed here, we should try to ensure that it "just builds". That will help not only on Windows.

[fanquake]

When cross-compiling on Linux, it may happen that configure believes that we're not cross-compiling because it's possible to execute windows binaries with an installed Wine and binfmt_misc enables. (This happens on my system but not on CI). See https://stackoverflow.com/questions/13150631/cross-compiling-why-checking-whether-we-are-cross-compiling-no#comment40154987_13150651. This by itself is not an issue: There's per se nothing wrong with configure building gen_context.exe and using wine to execute it. It's a little bit cumbersome though and together with the previous item that means that the gen_context invocation will fail.

I just wasted some time trying to figure out why my secp256k1 build suddenly "randomly" started failing to run gen_context (when compiling the master branch of Core), and it seems I ran into this issue. For the moment I've removed Wine, but that isn't really a solution, and for others running into this, the error is pretty obtuse. Weirdly, even after temporarily reinstalling Wine, I can't seem to recreate the failure (although didn't try too hard).

[real-or-random]

Everything has been done here, except for this item:

When using the library (i.e., SECP256K1_BUILD is not defined), one can declare the symbols explicitly using __declspec (dllimport). We currently don't do this because the only problem is a small overhead when calling a DLL function but the code still works. The reason to avoid __declspec (dllimport) is that static linking will fail. (I verified this by adding the declspec and configuring with --disable-shared.) Unfortunately there's no way of telling whether we'll link statically or not, so the best possible thing is to have a macro that the user can set. (See also sourceware.org/autobook/autobook/autobook_137.html, autotools.io/libtool/windows.html, #314).

• We currently don't have this macro. We could add it, maybe it's useful for someone and it won't create problems because it's not set by default.

This has been discussed at length in #314 and the decision was not to add the macro. I currently don't see a reason to reconsider this decision.

So this issue has been fully resolved. 🎉