Look for libraries in the LD_LIBRARY_PATH

[MakisH]

When building the adapter with preCICE as a static library, the paths for Boost and PETSc (and maybe also other dependencies in some cases) need to be set manually in the build script. However, some users already have these paths set in their LD_LIBRARY_PATH or LIBRARY_PATH and they should not need to specify them again. Moreover, this could lead into linking to the wrong version of a library, if multiple are installed.

We use the wmake build system, which is provided with OpenFOAM. However, this does not search directories like the LIBRARY_PATH, LD_LIBRARY_PATH or CPLUS_INCLUDE_PATH, as is described in this issue. One way to search them would be to add these directories in the LIB_LIBS and EXE_INC (for header files) of the Make/options file.

In the state of the adapter at commit e70e06c we had added these paths in the LIB_LIBS. However, for some users this triggered errors during linking:

g++ -std=c++11 -m64 -Dlinux64 -DWM_ARCH_OPTION=64 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -O3  -DNoRepository -ftemplate-depth-100 -I/data/scratch//OF-Adapter/OpenFOAM-5.0/src/finiteVolume/lnInclude -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/meshTools/lnInclude -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/transportModels/ -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/transportModels/compressible/lnInclude -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/thermophysicalModels/basic/lnInclude -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/TurbulenceModels/turbulenceModels/lnInclude -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/TurbulenceModels/compressible/lnInclude -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/TurbulenceModels/incompressible/lnInclude -I/data/scratch/<username>/precice/src/precice -I -I../ -DADAPTER_DEBUG_MODE -IlnInclude -I. -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/OpenFOAM/lnInclude -I/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/src/OSspecific/POSIX/lnInclude   -I/home/<username>/software/boost/include -I/usr/include/eigen3 -I  -fPIC -shared -Xlinker --add-needed -Xlinker --no-as-needed Make/linux64GccDPInt32Opt/Utilities.o Make/linux64GccDPInt32Opt/Interface.o Make/linux64GccDPInt32Opt/CouplingDataUser.o Make/linux64GccDPInt32Opt/CHT/Temperature.o Make/linux64GccDPInt32Opt/CHT/KappaEffective.o Make/linux64GccDPInt32Opt/CHT/HeatFlux.o Make/linux64GccDPInt32Opt/CHT/HeatTransferCoefficient.o Make/linux64GccDPInt32Opt/CHT/SinkTemperature.o Make/linux64GccDPInt32Opt/CHT/CHT.o Make/linux64GccDPInt32Opt/Adapter.o Make/linux64GccDPInt32Opt/preciceAdapterFunctionObject.o -L/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/platforms/linux64GccDPInt32Opt/lib \
    -lfiniteVolume -lmeshTools -lcompressibleTurbulenceModels -lincompressibleTurbulenceModels -L/home/<username>/software/boost/lib -L/data/scratch/<username>/software/petsc/arch-linux2-c-debug/lib -L  -L/data/scratch/<username>/OF-Adapter/ThirdParty-5.0/platforms/linux64Gcc/gperftools-svn/lib -L/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/platforms/linux64GccDPInt32Opt/lib/openmpi-system -L/data/scratch/<username>/OF-Adapter/ThirdParty-5.0/platforms/linux64GccDPInt32/lib/openmpi-system -L/usr/lib/openmpi/lib -L/home/<username>/OpenFOAM/<username>-5.0/platforms/linux64GccDPInt32Opt/lib -L/data/scratch/<username>/OF-Adapter/site/5.0/platforms/linux64GccDPInt32Opt/lib -L/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/platforms/linux64GccDPInt32Opt/lib -L/data/scratch/<username>/OF-Adapter/ThirdParty-5.0/platforms/linux64GccDPInt32/lib -L/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/platforms/linux64GccDPInt32Opt/lib/dummy -L/home/<username>/software/boost/lib -L/data/scratch/<username>/precice/build/last -L/data/scratch/<username>/software/petsc/arch-linux2-c-debug/lib  -L/data/scratch/<username>/precice/build/last -L -L -lprecice -lpetsc -lmpi_cxx -lboost_system -lboost_filesystem -lboost_log -lboost_log_setup -lboost_thread -lboost_program_options -lpthread -lpython2.7 -L -lyaml-cpp  -o /home/<username>/OpenFOAM/<username>-5.0/platforms/linux64GccDPInt32Opt/lib/libpreciceAdapterFunctionObject.so
/usr/bin/ld: Make/linux64GccDPInt32Opt/Utilities.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
Make/linux64GccDPInt32Opt/Utilities.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
/data/scratch/<username>/OF-Adapter/OpenFOAM-5.0/wmake/makefiles/general:167: recipe for target '/home/<username>/OpenFOAM/<username>-5.0/platforms/linux64GccDPInt32Opt/lib/libpreciceAdapterFunctionObject.so' failed
make: *** [/home/<username>/OpenFOAM/<username>-5.0/platforms/linux64GccDPInt32Opt/lib/libpreciceAdapterFunctionObject.so] Error 1

We need to investigate this issue further. Since it was always working for me, I need some help to reproduce the error or locate the problem.

[MakisH]

I created the branch debug_build to easily test this. I think I did some progress in the commit 8c8dbd1. See the commit message for details. In short, it looks like some empty -I and -L led to weird behavior.

In particular, an empty -I would make the linker not recognise the -fPIC flag, because I had put the ADAPTER_GLOBAL_CPLUS_INCLUDE_PATHS at the end of the include paths (in the SYS_INC), just before the -fPIC.

This originates from sequences of the form ":path", "path:", and "path::path" in the respective environment variables.

Please test and tell me if it is fixed now for your setups.

[uekerman]

It works for me now!
@floli ?

[MakisH]

That's very good news! Now, should we keep this version or the one with the paths specified in the script? Or a mixture of both?

The one you are testing now should work with the dependencies installed either in a system path, or in a path included in the respective variables. This requires that the user actually uses these variables.

The one in the master branch should work always, but requires the user to edit the script.

A mixture could use both pahts. But in which order?

[floli]

I can confirm that the debug_build as well as the master branch builds fine at our neon machine out of the box (no modifications)

Boost is installed in a non-standard path, with environment variables set, yaml-cpp is installed by the package manager.

[MakisH]

Thank you for testing this! I actually think we should keep this version only, since eitherways all the dependencies should be in some system-wide library paths also at runtime.

What I mean: if a user e.g. installs yaml-cpp from source in a non-standard directory and specifies the paths in the script, but not in LD_LIBRARY_PATH, then the adapter will compile, but it will not run. Actually, the ldd -r libpreciceAdapterFunctionObject.so will return undefined symbols, even though wmake will exit normally.