Segmentation fault when uploading to blob on arm32

[Nathan-Jordan]

ARM32 controller armv7.
Cross Compiled on Ubuntu-17 4.13.0-46-generic using:

• arm-linux-gnueabihf-gcc (Ubuntu/Linaro 7.2.0-6ubuntu1) 7.2.0

Cross Compiled Statically Linked:

• Azure-iot-sdk-c 1.11.0
• OpenSSL 3.2.1
• Curl 8.6.0
• zlib 1.3.1

Compiling only the iothub_client_sample_upload_to_blob sample on my Linux VM works and it successfully uploads (with just the azure sdk compiled). However when I cross compile the same code to my arm32 controller I get a segmentation fault when 'IoTHubDeviceClient_LL_UploadToBlob' is called.

Log output:

• Host ###:### was resolved.
• IPv6: (none)
• IPv4: ###
• Trying ###...
• Connected to ### port ###
• ALPN: curl offers http/1.1
• CAfile: /etc/ssl/certs/ca-certificates.crt
• CApath: /etc/ssl/certs
• SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 / x25519 / RSASSA-PSS
• ALPN: server did not agree on a protocol. Uses default.
• Server certificate:
• subject: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; CN=*.azure-devices.net
• start date: Feb 13 00:00:00 2024 GMT
• expire date: Jul 13 23:59:59 2024 GMT
• subjectAltName: host ### matched cert's "*.azure-devices.net"
• issuer: C=US; O=Microsoft Corporation; CN=MSFT RS256 CA-1
• SSL certificate verify ok.
• Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
• Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
• Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
• using HTTP/1.x

POST /devices/###/files/?api-version=2016-11-14 HTTP/1.1
Host: ###
Content-Type: application/json
Accept: application/json
User-Agent: iothubclient/1.11.0
Authorization: SharedAccessSignature sr=###
Content-Length: 41

< HTTP/1.1 200 OK
< content-length: 392
< content-type: application/json
< date: Tue, 27 Feb 2024 16:28:08 GMT
<

• Connection #0 to host ### left intact
  Segmentation fault

[ericwolz]

Is there a callstack??

[Nathan-Jordan]

[New Thread 0xb6ffc2f0 (LWP 5687)]

Thread 3 "Test" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb6ffc2f0 (LWP 5687)]
0xb67db48c in internal_getent (stream=0xb6300470, result=result@entry=0xb6ffb65c,
buffer=buffer@entry=0xb6ffb838 "gateway-prod-gw-northeur127.0.0.1 localhost Moxa\n",
buflen=buflen@entry=1024, errnop=errnop@entry=0xb6ffb74c, herrnop=herrnop@entry=0xb6ffb75c,
af=af@entry=0, flags=flags@entry=0) at nss_files/files-XXX.c:251
251 nss_files/files-XXX.c: No such file or directory.

(gdb) backtrace
#0 0xb67db48c in internal_getent (stream=0xb6300470, result=result@entry=0xb6ffb65c, buffer=buffer@entry=0xb6ffb838 "gateway-prod-gw-northeur127.0.0.1 localhost Moxa\n",
buflen=buflen@entry=1024, errnop=errnop@entry=0xb6ffb74c, herrnop=herrnop@entry=0xb6ffb75c, af=af@entry=0, flags=flags@entry=0) at nss_files/files-XXX.c:251
#1 0xb67dbeac in _nss_files_gethostbyname4_r (name=0x427088 "###", pat=0xb6ffb748,
buffer=0xb6ffb838 "gateway-prod-gw-northeur127.0.0.1 localhost Moxa\n", buflen=1024, errnop=0xb6ffb74c, herrnop=0xb6ffb75c, ttlp=0x0) at nss_files/files-hosts.c:392
#2 0x002b94a0 in gaih_inet.constprop ()
#3 0x002ba04a in getaddrinfo ()
#4 0x00079c16 in Curl_getaddrinfo_ex ()
#5 0x00071a04 in getaddrinfo_thread ()
#6 0x0007a5cc in curl_thread_create_thunk ()
#7 0x0028d96a in start_thread (arg=0x0) at pthread_create.c:458
#8 0x002bc022 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

May not be relavant but I was trying valgrind and it said this.
==3842== Massif, a heap profiler
==3842== Copyright (C) 2003-2015, and GNU GPL'd, by Nicholas Nethercote
==3842== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==3842== Command: ./Test
==3842==

valgrind: m_transtab.c:2459 (vgPlain_init_tt_tc): Assertion 'sizeof(TTEntryC) <= 88' failed.
Segmentation fault

[ASeidelt]

This might be related to #2554 ?

AFAIK this was fixed in the Repo, but there is no updated release yet

[ericwolz]

Can you verify the main branch?

[Nathan-Jordan]

Seemed to have fixed it, I was statically linking the libs. Dynamically linking fixed the issue

Found various mentions of this in similar posts:
https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1613067
https://sourceforge.net/p/iperf2/tickets/83/
https://sourceware.org/bugzilla/show_bug.cgi?id=10652
https://sourceware.org/bugzilla/show_bug.cgi?id=19341

After this I still had a crash with:

• SSL: couldn't create a context: error:140A90A1:lib(20):func(169):reason(161)
  and
  curl_easy_perform() failed: out of memory

Adding the below code fixed this:
#include <curl/curl.h>
curl_global_init(CURL_GLOBAL_SSL);

Ref:
https://stackoverflow.com/questions/166617/curle-out-of-memory-error-when-posting-via-https