SegmentationFault at install point NetCDF=0.10.3

[valeriupredoi]

Hi folks, we've started seeing perfectly reproducible SegFaults when trying to install NetCDF=0.10.3, either using a Project.toml or simply by installing it in from command line:

julia -e 'using Pkg; Pkg.add("NetCDF")'

that returns:

...
  [14a3606d] + MozillaCACerts_jll v2023.1.10
  [4536629a] + OpenBLAS_jll v0.3.23+4
  [bea87d4a] + SuiteSparse_jll v7.2.1+1
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850b90] + libblastrampoline_jll v5.8.0+1
  [8e850ede] + nghttp2_jll v1.52.0+1
  [3f19e933] + p7zip_jll v17.4.0+2
        Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`
    Building Conda ─→ `~/miniconda3/envs/esmvaltool/share/julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/b19db3927f0db4151cb86d073689f2428e524576/build.log`
    Building NetCDF → `~/miniconda3/envs/esmvaltool/share/julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/dda7b49694d15d7d1a1ba1d7c2a9142777028bd5/build.log`

[4351] signal (11.1): Segmentation fault
in expression starting at /home/runner/work/ESMValTool/ESMValTool/esmvaltool/install/Julia/setup.jl:10
Curl_hash_pick at /home/runner/miniconda3/envs/esmvaltool/bin/../lib/julia/../libcurl.so.4 (unknown line)
unknown function (ip: 0x7ff753272c8f)
Allocations: 10052000 (Pool: 10038065; Big: 13935); GC: 21

Please see GA test for full install stdout

• Python 3.11.10 in a miniconda3 environment conda 24.7.1
• julia version 1.9.3 from conda-forge
• OS: Ubuntu 24.04.1 LTS

In fairness, I am not 100% sure this is indeed from NetCDF, and could be this issue JuliaLang/Downloads.jl#260 - but thought of reporting it nonetheless, apologies if this is something else!

Please let me know if you need more details about the particularities of our system! Cheers very much for looking into it 🍺

[visr]

Hmm there seems to be quite a lot going on here. I am not sure we can do much in this repository. So this is using an old release of NetCDF.jl from 2020, that triggers a segfault when starting it in a Conda environment.

The netCDF build that comes with NetCDF.jl has improved a lot since 2020. I guess that old build may be clashing with the libcurl that is used by Conda? As you can see in issues like JuliaPy/PythonCall.jl#519 there can be clashes like this sometimes, and one solution is to pin Conda dependencies to versions that go well with the Julia environment. Though I'd try to use a more recent NetCDF.jl version if possible.

Newer netCDF builds will also be compatible with newer curl versions, by updates in Yggdrasil, like JuliaPackaging/Yggdrasil#7127.

[valeriupredoi]

Hi @visr and very many thanks for taking the time to reply! We have tested with a new NetCDF=0.12.0 and new Julia=1.10.4, but we are still hitting the curl-related SegFault but admittedly not from installing NetCDF AFAIK, please see our Github Action test - me not knowing much about Julia doesn't come in handy here, unfortunately - what do you recommend for a working environment in terms of packages and versions, in your opinion, please? Sorry for hassling you - I finally found someone who knows Julia and can help, would buy pints for that 🍺 😁

[valeriupredoi]

here's how our Project.toml looks like (RainFARM commented out since I know for certain that pulls in the old NetCDF):

ArgParse = "c7e460c6-2fb9-53a9-8c5b-16f535851c63"
Compat = "34da2185-b29b-5c13-b0c7-acf172513d20"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
NetCDF = "30363a11-5582-574a-97bb-aa9a979735b9"
PyCall = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0"
PyPlot = "d330b81b-6aea-500a-939a-2ce795aea3ee"
# RainFARM = "e9a4e08f-a0a3-5224-a821-6d0231c12d6b"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
YAML = "ddb6d928-2868-570f-bddf-ab3f9cf99eb6"

Many thanks again!

[valeriupredoi]

UPDATE: @visr it seems that all our and others' troubles stem from one single place (not NetCDF.jll not other bits) - the new curl 8.10+ - pinning it to lower than that allows for all Julia gubbins be installed even with the old Julia 1.9.3 - see our PR ESMValGroup/ESMValTool#3755 and Github Action - my apologies for bugging you here (and, invariably, for pointing a finger at NetCDF 😁 ). Many thanks for all your help! 🍺

[visr]

Thanks for letting us know, glad you can work around it. So it seems a change in curl 8.10 triggered a segfault through JuliaLang/Downloads.jl#260.

[valeriupredoi]

yep - and not being fluent in Julia, it wasn't obvious to me 😁