Android JNI Deadlock

[sjg-wdw]

We have an ANR turning up on Android. That's when the system kills the app for not responding.

It appears that we're stuck in some JNI related logic on a lock.

The main thread looks like this:
  #00  pc 0x000000000004b50c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
  #01  pc 0x00000000001b07a4  /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+148)
  #02  pc 0x00000000003a4294  /apex/com.android.art/lib64/libart.so (art::JNI<false>::CallObjectMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+500)
  #03  pc 0x00000000004023a4 /lib/arm64/libmapbox-gl.so
  #04  pc 0x000000000043e8a0 /lib/arm64/libmapbox-gl.so (jni::WeakReference<jni::Object<mbgl::android::NativeMapView>, jni::DefaultRefDeleter>::get(_JNIEnv&)+88)
  #05  pc 0x000000000043f570  /lib/arm64/libmapbox-gl.so (mbgl::android::NativeMapView::onWillStartRenderingFrame()+84)

And other threads also appear to be in similar locks:
"Jit thread pool worker thread 0" tid=12 Native
  #00  pc 0x000000000004b50c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
  #01  pc 0x00000000001b07a4  /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+148)
  #02  pc 0x00000000005d4a00  /apex/com.android.art/lib64/libart.so (art::ThreadPool::GetTask(art::Thread*)+120)
  #03  pc 0x00000000005d3c88  /apex/com.android.art/lib64/libart.so (art::ThreadPoolWorker::Run()+144)
  #04  pc 0x00000000005d3748  /apex/com.android.art/lib64/libart.so (art::ThreadPoolWorker::Callback(void*)+192)
  #05  pc 0x00000000000b0bd8  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
  #06  pc 0x00000000000505d0  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

  #00  pc 0x000000000004b50c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
  #01  pc 0x00000000001b07a4  /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+148)
  #02  pc 0x00000000003a599c  /apex/com.android.art/lib64/libart.so (art::JNI<false>::CallBooleanMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+500)
  #03  pc 0x000000000009b73c  /system/lib64/libandroid_runtime.so (_JNIEnv::CallBooleanMethod(_jobject*, _jmethodID*, ...)+124)
  #04  pc 0x0000000000123d64  /system/lib64/libandroid_runtime.so (JavaBBinder::onTransact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+156)
  #05  pc 0x0000000000049860  /system/lib64/libbinder.so (android::BBinder::transact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+232)
  #06  pc 0x00000000000523a0  /system/lib64/libbinder.so (android::IPCThreadState::executeCommand(int)+1032)
  #07  pc 0x0000000000051ee4  /system/lib64/libbinder.so (android::IPCThreadState::getAndExecuteCommand()+156)
  #08  pc 0x0000000000052724  /system/lib64/libbinder.so (android::IPCThreadState::joinThreadPool(bool)+60)
  #09  pc 0x0000000000078b90  /system/lib64/libbinder.so (android::PoolThread::threadLoop()+24)
  #10  pc 0x000000000001567c  /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+260)
  #11  pc 0x00000000000a0dac  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+140)
  #12  pc 0x0000000000014f14  /system/lib64/libutils.so (thread_data_t::trampoline(thread_data_t const*)+412)
  #13  pc 0x00000000000b0bd8  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
  #14  pc 0x00000000000505d0  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

"Picasso-Stats" tid=99 Native
  #00  pc 0x000000000004b50c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
  #01  pc 0x00000000001b07a4  /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+148)
  #02  pc 0x0000000000673d78  /apex/com.android.art/lib64/libart.so (art::GoToRunnable(art::Thread*)+480)
  #03  pc 0x0000000000673b54  /apex/com.android.art/lib64/libart.so (art::JniMethodEnd(unsigned int, art::Thread*)+28)
  at android.os.MessageQueue.nativePollOnce (Native method)
  at android.os.MessageQueue.next (MessageQueue.java:335)
  at android.os.Looper.loop (Looper.java:183)
  at android.os.HandlerThread.run (HandlerThread.java:67)

"Picasso-Dispatcher" tid=136 Native
  #00  pc 0x000000000004b50c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
  #01  pc 0x00000000001b07a4  /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+148)
  #02  pc 0x0000000000673d78  /apex/com.android.art/lib64/libart.so (art::GoToRunnable(art::Thread*)+480)
  #03  pc 0x0000000000673b54  /apex/com.android.art/lib64/libart.so (art::JniMethodEnd(unsigned int, art::Thread*)+28)
  at android.os.MessageQueue.nativePollOnce (Native method)
  at android.os.MessageQueue.next (MessageQueue.java:335)
  at android.os.Looper.loop (Looper.java:183)
  at android.os.HandlerThread.run (HandlerThread.java:67)

[sjg-wdw]

This is a call from JNI back into the Java side. We're not clear on what lock it's trying to grab.