[BUG] error: use of undeclared identifier 'strtouq' / 'strtoq'

[JMLX42]

Description

Trying to compile PCRE 8.45, I got the following error:

1. error: use of undeclared identifier 'strtoq'
2. error: use of undeclared identifier 'strtouq'

Here is the complete command line:

/opt/android-ndk-linux/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=armv7-none-linux-androideabi21 --sysroot=/opt/android-ndk-linux/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_CONFIG_H -I/srv/embedb/build/third_party/pcre/src/pcre-build -I/srv/embedb/build/third_party/pcre/src/pcre -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -Wformat -Werror=format-security   -fPIC -MD -MT CMakeFiles/pcrecpp.dir/pcrecpp.cc.o -MF CMakeFiles/pcrecpp.dir/pcrecpp.cc.o.d -o CMakeFiles/pcrecpp.dir/pcrecpp.cc.o -c /srv/embedb/build/third_party/pcre/src/pcre/pcrecpp.cc
/srv/embedb/build/third_party/pcre/src/pcre/pcrecpp.cc:885:17: error: use of undeclared identifier 'strtoq'; did you mean 'strtol'?
  long long r = strtoq(str, &end, radix);
                ^~~~~~
                strtol
/opt/android-ndk-linux/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/stdlib.h:91:6: note: 'strtol' declared here
long strtol(const char* __s, char** __end_ptr, int __base);
     ^
/srv/embedb/build/third_party/pcre/src/pcre/pcrecpp.cc:917:26: error: use of undeclared identifier 'strtouq'; did you mean 'strtoul'?
  unsigned long long r = strtouq(str, &end, radix);
                         ^~~~~~~
                         strtoul
/opt/android-ndk-linux/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/stdlib.h:93:15: note: 'strtoul' declared here
unsigned long strtoul(const char* __s, char** __end_ptr, int __base);
              ^
2 errors generated.

Affected versions

r23, r24, r25

Canary version

No response

Host OS

Linux

Host OS version

Ubuntu 20.04

Affected ABIs

armeabi-v7a

Build system

CMake

Other build system

No response

minSdkVersion

19

Device API level

No response

[JMLX42]

The following works:

mkdir build
cd build
cmake .. \
  -DANDROID_PLATFORM=android-19
  -DCMAKE_BUILD_TYPE=Release \
  -DANDROID_ABI=armeabi-v7a \
  -DANDROID_STL=c++_shared
make

The following does not work:

mkdir build
cd build
cmake .. \
  -DANDROID_PLATFORM=android-24
  -DCMAKE_BUILD_TYPE=Release \
  -DANDROID_ABI=armeabi-v7a \
  -DANDROID_STL=c++_shared
make

[JMLX42]

The following works:

mkdir build
cd build
cmake .. \
  -DANDROID_PLATFORM=android-19
  -DCMAKE_BUILD_TYPE=Release \
  -DANDROID_ABI=armeabi-v7a \
  -DANDROID_STL=c++_shared
make

The following does not work:

mkdir build
cd build
cmake .. \
  -DANDROID_PLATFORM=android-24
  -DCMAKE_BUILD_TYPE=Release \
  -DANDROID_ABI=armeabi-v7a \
  -DANDROID_STL=c++_shared
make

Apparently, is works from android-19 up to android-20.

[Grimler91]

pcre's CMakeLists.txt uses check_function_exists to check if strtoq is available. The documentation for that macro lists several shortcomings, and advises to use cmake_symbol_exists instead. pcre builds fine if cmake_symbol_exists is used instead, or if we skip cmake altogether and use the configure && make scripts.

So, don't think this is a ndk issue, rather cmake/pcre issue.

[JMLX42]

@Grimler91 thank you! Any reason why it still works up to android-20 though?

[Grimler91]

@Grimler91 thank you! Any reason why it still works up to android-20 though?

Don't know, haven't tested building with an older ndk. Maybe ndk developers have an idea

[enh-google]

@Grimler91 thank you! Any reason why it still works up to android-20 though?

Don't know, haven't tested building with an older ndk. Maybe ndk developers have an idea

i can kind of answer this...

basically, strtoq() and strtouq() are in bionic, and have been since API 21. (despite the fact that they're not in any header file.)

i don't know anything about cmake, but i suspect the difference there is that check_function_exists is looking for the symbol in the ELF file (which succeeds, correctly) while cmake_symbol_exists is looking for the declaration in the header file (which fails, also correctly).

obviously the real question though is "why are they in libc.so but not in any header file?", and that's where i don't have a really satisfactory answer because i don't find any explicit record of any decisions in the history. as far as i can tell, the symbol was added by accident (possibly because we didn't even realize it wasn't in any header, and because it is available on macOS/iOS, so i may have thought "oh, that might be useful for developers" without questioning who in their right mind wouldn't just use strtoll() instead) and then when we discovered that we had the symbol but not the declaration (while i was cleaning up our strto*() functions for other reasons), we left it like that because we couldn't really justify the disruption of adding it the headers.

[JMLX42]

@enh-google thanks that makes a lot of sense.

obviously the real question though is "why are they in libc.so but not in any header file?"

Would it make sense to add it to the headers then?

[DanAlbert]

obviously the real question though is "why are they in libc.so but not in any header file?"

Would it make sense to add it to the headers then?

It sounds like we should either add it to the header or remove it from the stub library. It sounds like it's probably better to just add it to the header, but if we for some reason don't want to make this available (I'm not familiar with the API, but if all we're doing is giving you yet-another-footgun, might be better to remove entirely), we should also remove it from the stubs. Being inconsistent is likely the worst option.

[enh-google]

obviously the real question though is "why are they in libc.so but not in any header file?"

Would it make sense to add it to the headers then?

It sounds like we should either add it to the header or remove it from the stub library. It sounds like it's probably better to just add it to the header, but if we for some reason don't want to make this available (I'm not familiar with the API, but if all we're doing is giving you yet-another-footgun, might be better to remove entirely), we should also remove it from the stubs. Being inconsistent is likely the worst option.

it's not a footgun; it's just stupid. it's just non-standard names for strtoll/strtoull with an annoying return type that's an otherwise unused synonym for long long with the misleading name quad_t. (although checking macOS shows that they've undone that part and just used long long instead, even though they haven't updated their man page to show this divergence from BSD.)

i've asked serbanc whether this is actually used by anything in the Play Store, but since the answer to that question is basically always "yes", i'm working on resigning myself to adding this to the header. it's likely to break someone who's defined strtoq() (perhaps for 128-bit, which would at least be useful!), but, yeah, being inconsistent is bad.

[JMLX42]

i'm working on resigning myself to adding this to the header

AFAIK in this very case, PCRE has a fallback:

#if defined HAVE_STRTOQ
  long long r = strtoq(str, &end, radix);
#elif defined HAVE_STRTOLL
  long long r = strtoll(str, &end, radix);
#elif defined HAVE__STRTOI64
  long long r = _strtoi64(str, &end, radix);
#elif defined HAVE_STRTOIMAX
  long long r = strtoimax(str, &end, radix);
#else
#error parse_longlong_radix: cannot convert input to a long-long
#endif

So I guess HAVE_STRTOLL would just work in this case assuming it is in the headers and in the lib (AFAIK it is because CMake's check_function_exists finds it). Correct?

i'm working on resigning myself to adding this to the header

Any reason why those functions have been (apparently) removed from android-21+?

Can't we just revert the corresponding commit?

[enh-google]

Any reason why those functions have been (apparently) #1803 (comment)?

no, i think you're reading that backwards ... or my understanding of reality is fundamentally broken :-)

i think these are the facts:

1. these functions are not in libc.so for API levels <= 20.

2. they are in libc.so for API levels >= 21.

3. they are not in the headers for any API level.

the weird thing you're probably missing here is that the libc.so files in the NDK aren't real, and nor are they based on the ELF files that actually ship as part of the OS. they're based on a text file that describes -- or claims to, anyway -- what was in every release.

afaict from this bug report (that is "unless you know something else that you haven't told me yet" :-) ), that text file (and thus the generated "fake" libc.so files in the NDK) are indeed correct. aiui what's changed here is that you've changed your build from targeting an API level that didn't have this function to one that did, and PCRE's "check the ELF file" test for symbol presence (which is a bad idea for several reasons, only one of which is this one) now sees it, but trips up when it tries to use it because this function has never had a declaration in any bionic header file.

---

anyway, it turns out that there are no apps in the Play Store that refer to this symbol. so i'll upload a change today to remove it entirely, and even projects that do (misguidedly) look in ELF files for symbols rather than trying compile tests will "just work" once that makes its way into a released NDK...

[enh-google]

https://android-review.googlesource.com/c/platform/bionic/+/2295477 removes the aliases from bionic.

[DanAlbert]

Change is merged into bionic. I'll update the sysroot for r25 when we get a new toolchain so this will ship in r25c.