Nightly build for Windows

[jdblischak]

This nightly build for Windows is a work-in-progress. The existing GitHub Actions pipeline to test TileDB-VCF against release libtiledb on Windows relies exclusively on conda (ie everything is built inside of an activated conda environment). In contrast, for this build against dev libtiledb, I have tried to use conda as little as possible (currently only for installing the Python dependencies, but I'm open to trying to replace these with PyPI packages). Unfortunately this is causing problems. Code that runs fine inside a conda env doesn't work in a more minimal setup. I'd appreciate any advice.

Problem 1: Unable to run tiledbvcf.exe version

I install tiledbvcf.exe into a local directory that is not on the PATH. Thus I provide the full path when trying to execute it. This does not work, and I've tried various things (with and without .exe, absolute versus relative path). This approach of installing into a local directory works fine on Linux/macOS. Is this even possible on Windows?

D:\a\TileDB-VCF\TileDB-VCF\install\bin\tiledbvcf.exe version 

Problem 2: Python client unable to find tiledbvcf.dll at runtime

I can build the Python client against the locally built libtiledbvcf using --libtiledbvcf. However, it immediately fails at runtime because it can't find tiledbvcf.dll (or one of its dependencies)

python -c "import tiledbvcf; print(tiledbvcf.version)" 
Traceback (most recent call last):
  File "d:\a\tiledb-vcf\tiledb-vcf\tiledb-vcf\apis\python\src\tiledbvcf\__init__.py", line 21, in _load_libs
    ctypes.CDLL(os.path.join(lib_dir, lib_name))
  File "C:\Users\runneradmin\micromamba\envs\py4vcf\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'd:\a\tiledb-vcf\tiledb-vcf\tiledb-vcf\apis\python\src\tiledbvcf\tiledbvcf.dll' (or one of its dependencies). Try using the full path with constructor syntax.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "d:\a\tiledb-vcf\tiledb-vcf\tiledb-vcf\apis\python\src\tiledbvcf\__init__.py", line 28, in <module>
    _load_libs()
  File "d:\a\tiledb-vcf\tiledb-vcf\tiledb-vcf\apis\python\src\tiledbvcf\__init__.py", line 24, in _load_libs
    ctypes.CDLL(lib_name)
  File "C:\Users\runneradmin\micromamba\envs\py4vcf\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'tiledbvcf.dll' (or one of its dependencies). Try using the full path with constructor syntax.

I've already added the installation location to PATH. Maybe the issue is that tiledbvcf.dll can't find tiledb.dll?

set PATH=%GITHUB_WORKSPACE%\install\bin;%PATH%
echo "PATH: %PATH%"

Problem 3: What tests to run for libtiledbvf?

The existing Windows build only runs the tests for the Python client. Which libtiledbvcf tests are feasible to run on Windows?

The unit tests, tiledb_vcf_unit, require building an executable and then running it. But from the example of tiledbvcf.exe version above, I don't think I can simply execute tiledb_vcf_unit like I do for Linux/macOS.

The script run-cli-tests.sh requires bcftools to be installed, and I don't think it's worth the effort to build bcftools on Windows.

[ihnorton]

Per discussion, let's check CMAKE_INSTALL_LIBDIR, it is possible that the DLLs are being installed in lib where they can't be found (no rpath on windows, they need to go in bin/.

[jdblischak]

Quick update. I confirmed that the conda env installs both tiledbvcf.exe and tiledbvcf.dll into Library\bin, which explains why it works. However, it's unclear to me how this is configured. The conda recipe build script doesn't set any extra flags to control this. I also checked the compiler feedstock, which defines various CMake env vars. It doesn't define CMAKE_INSTALL_LIBDIR. The most promising one I found was CMAKE_PROGRAM_PATH.

Regardless, I pushed a commit to try CMAKE_INSTALL_LIBDIR

rem confirming conda installation setup
mamba create -n win-vcf -c tiledb tiledbvcf-py
mamba activate win-vcf
dir %CONDA_PREFIX%\Library\bin\tile*
rem  Volume in drive C has no label.
rem  Volume Serial Number is 8038-C77D
rem 
rem  Directory of C:\Users\john\mambaforge\envs\win-vcf\Library\bin
rem 
rem 11/03/2023  09:51 AM        15,554,048 tiledb.dll
rem 11/10/2023  08:21 PM         1,328,640 tiledbvcf.dll
rem 11/10/2023  08:21 PM         1,752,576 tiledbvcf.exe
rem 11/10/2023  08:21 PM         2,159,104 tiledbvcf4test.dll
rem                4 File(s)     20,794,368 bytes
rem                0 Dir(s)  27,726,221,312 bytes free
tiledbvcf version
rem TileDB-VCF version
rem TileDB version 2.17.4
rem htslib version 1.15.1

[jdblischak]

Setting CMAKE_INSTALL_LIBDIR had no effect. Furthermore, I realized that the .dll was already being installed in bin alongside the .exe. From the build log:

    -- Install configuration: "Release"
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/hts-3.lib
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/spdlog.lib
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/bin/tiledbvcf.exe
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/tiledbvcf.lib
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/bin/tiledbvcf.dll
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/tiledbvcf.h
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/tiledbvcf_enum.h
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/arrow.h
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/include/tiledbvcf/tiledbvcf_export.h
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/lib/tiledbvcf4test.lib
    -- Installing: D:/a/TileDB-VCF/TileDB-VCF/install/bin/tiledbvcf4test.dll
    -- Up-to-date: D:/a/TileDB-VCF/TileDB-VCF/install/lib/hts-3.lib

Thus I thought maybe I need to set some env var, but confusingly, locally I can run tiledbvcf by providing the path even when the conda env isn't activated. In other words, I don't see the difference between the example below and what I am trying to accomplish in the CI job

(base) C:\Users\john>mambaforge\envs\win-vcf\Library\bin\tiledbvcf version
rem TileDB-VCF version
rem TileDB version 2.17.4
rem htslib version 1.15.1

[jdblischak]

I uploaded the installation directory (contains both libtiledb and libtiledbvcf) as an artifact. It can be downloaded by scrolling to the bottom of the latest CI build

Note that I also run dumpbin in the pipeline, eg here