chore: update matrix to macOS-latest #3164
Conversation
``` Run Tests (macos-11, 3.8, x64, full) This is a scheduled macOS-11 brownout. The macOS-11 environment is deprecated and will be removed on June 28th, 2024. Run Tests (macos-11, 3.11, x64, full) GitHub Actions has encountered an internal error when running your job. Run Tests (macos-11, 3.9, x64, full) GitHub Actions has encountered an internal error when running your job. Run Tests (macos-11, 3.10, x64, full) GitHub Actions has encountered an internal error when running your job. Run Tests (macos-11, 3.12, x64, full) This is a scheduled macOS-11 brownout. The macOS-11 environment is deprecated and will be removed on June 28th, 2024. Run Tests (macos-11, 3.12, x64, full) GitHub Actions has encountered an internal error when running your job. ```
There was a problem hiding this comment.
@jpivarski - we need to remove the macOS-11 from required tests and add macOS-12 instead. I don't think, I have rights to do it.
Oh, macOS-12 build fails with:
==================================== ERRORS ====================================
____________ ERROR collecting tests/test_0002_minimal_listarray.py _____________
ImportError while importing test module '/Users/runner/work/awkward/awkward/tests/test_0002_minimal_listarray.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/python.py:492: in importtestmodule
mod = import_path(
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/pathlib.py:591: in import_path
importlib.import_module(module_name)
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1387: in _gcd_import
???
<frozen importlib._bootstrap>:1360: in _find_and_load
???
<frozen importlib._bootstrap>:1331: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:935: in _load_unlocked
???
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/assertion/rewrite.py:178: in exec_module
exec(co, module.__dict__)
tests/test_0002_minimal_listarray.py:8: in <module>
import awkward as ak
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/awkward/__init__.py:33: in <module>
import awkward.jax
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/awkward/jax.py:10: in <module>
from awkward import highlevel
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/awkward/highlevel.py:17: in <module>
from awkward_cpp.lib import _ext
E ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/awkward_cpp/lib/_ext.cpython-312-darwin.so, 0x0002): weak-def symbol not found (__ZN7awkward14ForthMachineOfIiiE11begin_againERKNSt3__13mapINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS2_10shared_ptrINS_16ForthInputBufferEEENS2_4lessIS9_EENS7_INS2_4pairIKS9_SC_EEEEEEb)
e.g. this one:
awkward::ForthMachineOf<int, int>::begin_again(std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::shared_ptr<awkward::ForthInputBuffer>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::shared_ptr<awkward::ForthInputBuffer> > > > const&, bool)
|
Updating to macOS-latest:
|
|
|
By updating from @henryiii, there was a time several months ago when GitHub Actions made a big jump in MacOS version, and some things were broken as a result. I think this was one of them: the linker wasn't right, and therefore libawkward.dylib symbols didn't get properly linked into _ext.*.dylib. (The Avro failure is just the canary in the coalmine: it's the first use of the _ext extension module, through AwkwardForth.) Is this familiar? Do you know what's happening here? |
Thanks! I thought it looked familiar :-) Yes, it's a linker issue. The -undefined dynamic_lookup flag allows the linker to defer symbol resolution until runtime, which can be problematic if the required symbols are not found. This is often used in Python extensions to allow them to be dynamically loaded. Chained fixups are a newer feature in macOS that optimize the way dynamic libraries are loaded. However, they may not always work correctly with -undefined dynamic_lookup. Unfortunately, my laptop is MacOS 11.6 and I'm planning to upgrade it to 12 this weekend so that I could test if we need to explicitly export the symbols and check if the |
|
My MacOS is 14.5 and I just ran another installation: no compilation issues, linker issues, or dynamic loading issues. All of the tests pass. (Major versions 11, 12, and 14 seem pretty far apart. I checked on endoflife.date/macos and MacOS 11 was dropped by Apple last September.) I think we were using an old Mac version because it works on all versions from the version used for compilation onward, and we're therefore covering all versions that are still in service. I don't know why it would fail to compile on GitHub's MacOS 11 and not on my MacOS 14. |
|
Apple was trying to move away from macos-latest (and macOS-14) are ARM, while macos-13 and before is Intel, on GHA. There are some issues with newer macOS versions supporting older ones around AVX instructions, I think, but it's not a linker issue, I think it was just a bug with the newest compilers at one point. |
|
Have you tried macos-13? |
I have tried the latest that was defaulted to 13. |
|
latest is 14 now, has been for a few weeks. That would be much faster, but is also a bigger change (Apple Silicon). 13 seems to segfault. |
See my comments above. I think, it was 13 😀 |
ianna
changed the title
|
@henryiii - I think, I'm still missing something. Could you, please have a look? Thanks! |
1 similar comment
|
@henryiii - I think, I'm still missing something. Could you, please have a look? Thanks! |
|
Test To follow-up, @ianna and I worked on reproducing the issue on macOS independently of the GitHub action, and we were able to reproduce the seg fault. System details: After building in a Python 3.1o env. following the instructions1, we can confirm that the test Footnotes |
|
I haven't been able to reproduce. Tried on Intel and AS. I had old CLT (11!), so was finally able to get it to see an update by touching rm -r awkward-cpp/build
git submodule update --init --recursive
nox -s prepare
uv venv
. .venv/bin/activate.fish
uv pip install pip pytest
pip install -v ./awkward-cpp
pip install -e .
pytest tests/test_1345_avro_reader.py$ sw_vers
ProductName: macOS
ProductVersion: 14.5
BuildVersion: 23F79
$ uname -a
Darwin [...] 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:16:51 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T8103 arm64
$ clang --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin$ sw_vers
ProductName: macOS
ProductVersion: 14.4.1
BuildVersion: 23E224
$ uname -a
Darwin [...] Darwin Kernel Version 23.4.0: Fri Mar 15 00:11:05 PDT 2024; root:xnu-10063.101.17~1/RELEASE_X86_64 x86_64
$ clang --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: x86_64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin |
|
Thanks to @henryiii who fixed the build on MacOS-13. |
|
For future reference, was the issue effectively fixed by the following commit and pull request ? You may also want to update the issue linked above (#3181) to indicate it was fixed by that same pull request |
The macOS-11 environment is deprecated and will be removed on June 28th, 2024. Currently, all tests on MacOS-11 are cancelled: