Error in test for xarray_adaptor.xiterator on MSVC x64

[emmenlau]

I'm running the test suite for xtensor on Microsoft Windows 7 x64 for a build with Visual Studio 2017 command line tools. The tests fail with the following message on the command line:

[ RUN      ] xarray_adaptor.xiterator
FAILED: test/CMakeFiles/xtest 

When run manually from the user interface, the attached popup message is shown that seems to indicate a vector out of bounds access.

Is this a problem on my side? anything I can do to help debug this issue? I can not easily run GUI applications on MSVC and I don't have Visual Studio GUI installed, so my debugging capabilities are quite limited.

[JohanMabille]

Can you enable xtensor asserts? This is done by defining XTENSOR_ENABLE_ASSERT either as a cmake command argument (-DXTENSOR_ENABLE_ASSERT=ON) or directly in the code (define it in xtensor_configuration.hpp), the rebuild and rerun the test suite. It should give you more insights.

[emmenlau]

Sorry, I should have mentioned my build flags. I build latest xtensor from 1hr ago (2288e86) with build instructions (leaving out irrelevant details like install paths):

export CPPFLAGS="${CPPFLAGS} /DDEBUG /DWINVER=_WIN32_WINNT_WIN7 /D_WIN32_WINNT=_WIN32_WINNT_WIN7"
export CFLAGS="${CFLAGS} /MDd /Zi ${CPPFLAGS}"
export CXXFLAGS="${CXXFLAGS} /MDd /Zi ${CPPFLAGS}"
export LDFLAGS="${LDFLAGS} /MACHINE:X64 /DEBUG"
cmake "D:/xtensor" \
    -DCMAKE_VERBOSE_MAKEFILE="ON" \
    -DCMAKE_BUILD_TYPE="Debug" \
    -DBUILD_SHARED_LIBS="OFF" \
    -DXTENSOR_ENABLE_ASSERT="ON" \
    -DXTENSOR_CHECK_DIMENSION="OFF" \
    -DDISABLE_VS2017=ON \
    -DBUILD_TESTS=ON \
    -DBUILD_BENCHMARK=OFF \
    -DCPP17="OFF" \
    -DXTENSOR_USE_XSIMD="ON" \
    -DXTENSOR_USE_TBB="OFF"

[JohanMabille]

Does it work if you build in Release mode? I'm afraid this error is due to additional non standard checks of iterators by Microsoft, even if they're not dereferenced.

You can also compile in debug with _ITERATOR_DEBUG_LEVEL defined to 0 to prevent iterator checks.

[emmenlau]

The test works fine in release mode. This is interesting, I did not know Microsoft has more "extensive" checks. Now I'm a bit lost what that means: it would be a bit unfortunate if parts of xtensor do work in debug mode. Is that the implied meaning of this error, or is the problem just with this specific test code (outside of xtensor)?

[JohanMabille]

So this error is the combination of two things:

• xtensor use past-the-end iterators that make it easy to step back into the container from the end, whatever the traversal order is. You can find more detail about this in Iterating over expressions. Since we never dereference these iterators, it is perfectly fine to do this.

• Microsoft has some additional, non standard, checks for iterators. Until recently these checks were enabled only when dereferencing the iterators, it seems that things have changed within VS2019. This is unfortuante since as long as you do not dereference an iterator or a pointer, it can have any value you like. The checks are controlled with _ITERATOR_DEBUG_LEVEL which is set to 0 (no check) in Release, and 1 or 2 (check enabled) in Debug.

I usually strongly encourage to disable these checks, they are most of the time totally useless, and may lead to build errors such as "'__iTERATOR8DEBUG_LEVEL': value '2' doesn't match value '0' in XXX" where XXX is an external dependency that is available in Release mode only.

[emmenlau]

I will do as you suggest and disable the checks. Is it possible for me to add this hint do the documentation (either FAQ or the build flags page)?

A tiny comment: I'm still on VS2017 Service Pack 4, so the problem seems to affect earlier versions of Visual Studio than VS2019 too.

[JohanMabille]

Indeed, adding this hint in the build flags page of the documentation would be a good idea.

My bad regarding the version of VS, indeed the error started with VS2017.