Double lock of a mutex in org.yb.cql.TestAuthentication#testAlterSuperuserForExistingRole

[mbautin]

Observed on a TSAN build with Clang 11 on Ubuntu 20.04

[2021-03-26T19:45:41 yb_build.sh:1382 main] Java build finished, total time information above.
[2021-03-26T19:45:41 common-test-env.sh:1853 resolve_and_run_java_test] Running Java test org.yb.cql.TestAuthentication#testAlterSuperuserForExistingRole
[2021-03-26T19:45:41 common-test-env.sh:1863 resolve_and_run_java_test] Java test class: org.yb.cql.TestAuthentication
[2021-03-26T19:45:41 common-test-env.sh:1864 resolve_and_run_java_test] Java test method name and optionally a parameter set index: testAlterSuperuserForExistingRole

• /home/yugabyteci/code/yugabyte-db/bin/repeat_unit_test.sh yb-cql org.yb.cql.TestAuthentication#testAlterSuperuserForExistingRole --java --build-type tsan --num-iter 10 --parallelism 1
  [2021-03-26T19:45:41 repeat_unit_test.sh:391 main] Using test parallelism of 1 (build type: tsan)
  [2021-03-26T19:45:41 repeat_unit_test.sh:395 main] YB_COMPILER_TYPE env variable was set to 'clang11' by the caller.
  [2021-03-26T19:45:41 repeat_unit_test.sh:397 main] gtest_filter is org.yb.cql.TestAuthentication#testAlterSuperuserForExistingRole (from the command line)
  [2021-03-26T19:45:41 repeat_unit_test.sh:403 main] Saving repeated test execution logs to: /home/yugabyteci/logs/repeat_unit_test/yb-cql/org.yb.cql.TestAuthentication#testAlterSuperuserForExistingRole/2021-03-26T19_45_41
  /home/yugabyteci/code/yugabyte-db/bin/repeat_unit_test.sh: line 293: ulimit: core file size: cannot modify limit: Operation not permitted
  [2021-03-26T19:50:56 common-test-env.sh:1157 did_test_succeed] Test failure reason: exit code: 1
  FAILED: iteration 1, 315 sec; test log path: /home/yugabyteci/logs/repeat_unit_test/yb-cql/org.yb.cql.TestAuthentication#testAlterSuperuserForExistingRole/2021-03-26T19_45_41/1.reports_combined_logs.txt.gz
  /home/yugabyteci/code/yugabyte-db/bin/repeat_unit_test.sh: line 293: ulimit: core file size: cannot modify limit: Operation not permitted

2021-03-26 19:47:55,946 (main) [INFO - org.yb.minicluster.MiniYBCluster.shutdownDaemons(MiniYBCluster.java:1030)] Waiting for PID 9172 for 540000 ms
ts2|pid9338|:30705|http://127.141.71.74:13264 ==================
ts2|pid9338|:30705|http://127.141.71.74:13264 WARNING: ThreadSanitizer: double lock of a mutex (pid=9338)
ts2|pid9338|:30705|http://127.141.71.74:13264 #0 pthread_mutex_lock (yb-tserver+0x45ccb6)
ts2|pid9338|:30705|http://127.141.71.74:13264 #1 std::__1::mutex::lock() (libc++.so.1+0xbec89)
ts2|pid9338|:30705|http://127.141.71.74:13264 #2 std::__1::lock_guardstd::__1::mutex::lock_guard(std::__1::mutex&) /home/yugabyteci/code/yugabyte-db-thirdparty/installed/tsan/libcxx/include/c++/v1/__mutex_base:91:27 (libyb_util.so+0x2a0828)
ts2|pid9338|:30705|http://127.141.71.74:13264 #3 yb::Synchronizer::StatusCB(yb::Status const&) /home/yugabyteci/code/yugabyte-db/build/tsan-clang11-dynamic-ninja/../../src/yb/util/async_util.cc:35:31 (libyb_util.so+0x54dc37)
ts2|pid9338|:30705|http://127.141.71.74:13264 #4 yb::tserver::(anonymous namespace)::LeaderMasterCallback(std::__1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&) /home/yugabyteci/code/yugabyte-db/build/tsan-clang11-
dynamic-ninja/../../src/yb/tserver/heartbeater.cc:246:14 (libtserver.so+0x21ccbe)
ts2|pid9338|:30705|http://127.141.71.74:13264 #5 yb::internal::RunnableAdapter<void ()(std::__1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&)>::Run(std::__1::shared_ptr<yb::tserver::(anonymous namespace)::FindLead
erMasterData> const&, yb::Status const&, yb::HostPort const&) /home/yugabyteci/code/yugabyte-db/build/tsan-clang11-dynamic-ninja/../../src/yb/gutil/bind_internal.h:303:12 (libtserver.so+0x22020c)
ts2|pid9338|:30705|http://127.141.71.74:13264 #6 yb::internal::InvokeHelper<false, void, yb::internal::RunnableAdapter<void ()(std::__1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&)>, void (std::__1::shared_ptr<yb
::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&)>::MakeItSo(yb::internal::RunnableAdapter<void ()(std::1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&)>, std::
1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&) /home/yugabyteci/code/yugabyte-db/build/tsan-clang11-dynamic-ninja/../../src/yb/gutil/bind_internal.h:920:14 (libtserver.so+0x22019c)
ts2|pid9338|:30705|http://127.141.71.74:13264 #7 yb::internal::Invoker<1, yb::internal::BindState<yb::internal::RunnableAdapter<void ()(std::__1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&)>, void (std::__1::shar
ed_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> const&, yb::Status const&, yb::HostPort const&), void (std::__1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData>)>, void (std::__1::shared_ptr<yb::tserver::(anonymous namespace)::FindLeaderMasterData> con
st&, yb::Status const&, yb::HostPort const&)>::Run(yb::internal::BindStateBase*, yb::Status const&, yb::HostPort const&) /home/yugabyteci/code/yugabyte-db/build/tsan-clang11-dynamic-ninja/../../src/yb/gutil/bind_internal.h:1222:12 (libtserver.so+0x220124)
ts2|pid9338|:30705|http://127.141.71.74:13264 #8 yb::Callback<void (yb::Status const&, yb::HostPort const&)>::Run(yb::Status const&, yb::HostPort const&) const /home/yugabyteci/code/yugabyte-db/build/tsan-clang11-dynamic-ninja/../../src/yb/gutil/callback.h:493:12 (libmaster_rpc.so+0x20
b56)
ts2|pid9338|:30705|http://127.141.71.74:13264 #9 yb::master::GetLeaderMasterRpc::GetMasterRegistrationRpcCbForNode(int, yb::Status const&, std::__1::shared_ptryb::rpc::RpcCommand const&, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>) /hom
e/yugabyteci/code/yugabyte-db/build/tsan-clang11-dynamic-ninja/../../src/yb/master/master_rpc.cc:311:14 (libmaster_rpc.so+0x1c68f)
ts2|pid9338|:30705|http://127.141.71.74:13264 #10 decltype((std::__1::forwardyb::master::GetLeaderMasterRpc*&(fp0)).fp(std::__1::forward<int&>(fp1), std::__1::forward<yb::Status const&>(fp1), std::__1::forward<std::__1::shared_ptryb::rpc::RpcCommand&>(fp1), std::__1::forward<boos
t::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>&>(fp1))) std::__1::__invoke<void (yb::master::GetLeaderMasterRpc::&)(int, yb::Status const&, std::__1::shared_ptryb::rpc::RpcCommand const&, boost::container::stable_vector_iterator<std::_1::shared
ptryb::rpc::RpcCommand, false>), yb::master::GetLeaderMasterRpc*&, int&, yb::Status const&, std::__1::shared_ptryb::rpc::RpcCommand&, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>&, void>(void (yb::master::GetLeaderMasterRpc::&)(int, yb::
Status const&, std::__1::shared_ptryb::rpc::RpcCommand const&, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>), yb::master::GetLeaderMasterRpc&, int&, yb::Status const&, std::__1::shared_ptryb::rpc::RpcCommand&, boost::container::stable_vec
tor_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>&) /home/yugabyteci/code/yugabyte-db-thirdparty/installed/tsan/libcxx/include/c++/v1/type_traits:3840:1 (libmaster_rpc.so+0x2c388)
ts2|pid9338|:30705|http://127.141.71.74:13264 #11 std::__1::__bind_return<void (yb::master::GetLeaderMasterRpc::)(int, yb::Status const&, std::__1::shared_ptryb::rpc::RpcCommand const&, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>), std
::__1::tuple<yb::master::GetLeaderMasterRpc, int, std::__1::placeholders::__ph<1>, std::__1::shared_ptryb::rpc::RpcCommand, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false> >, std::__1::tuple<yb::Status const&>, __is_valid_bind_return<void (yb:
:master::GetLeaderMasterRpc::)(int, yb::Status const&, std::__1::shared_ptryb::rpc::RpcCommand const&, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>), std::__1::tuple<yb::master::GetLeaderMasterRpc, int, std::__1::placeholders::__ph<1>, std
::__1::shared_ptryb::rpc::RpcCommand, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false> >, std::__1::tuple<yb::Status const&> >::value>::type std::__1::__apply_functor<void (yb::master::GetLeaderMasterRpc::)(int, yb::Status const&, std::__1::sha
red_ptryb::rpc::RpcCommand const&, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false>), std::__1::tuple<yb::master::GetLeaderMasterRpc, int, std::__1::placeholders::__ph<1>, std::__1::shared_ptryb::rpc::RpcCommand, boost::container::stable_vect
or_iterator<std::__1::shared_ptryb::rpc::RpcCommand, false> >, 0ul, 1ul, 2ul, 3ul, 4ul, std::__1::tuple<yb::Status const&> >(void (yb::master::GetLeaderMasterRpc::&)(int, yb::Status const&, std::__1::shared_ptryb::rpc::RpcCommand const&, boost::container::stable_vector_iterator<std::
__1::shared_ptryb::rpc::RpcCommand, false>), std::__1::tuple<yb::master::GetLeaderMasterRpc, int, std::__1::placeholders::__ph<1>, std::__1::shared_ptryb::rpc::RpcCommand, boost::container::stable_vector_iterator<std::__1::shared_ptryb::rpc::RpcCommand*, false> >&, std::__1::__tupl
e_indices<0ul, 1ul, 2ul, 3ul, 4ul>, std::__1::tuple<yb::Status const&>&&) /home/yugabyteci/code/yugabyte-db-thirdparty/installed/tsan/libcxx/include/c++/v1/functional:2853:12 (libmaster_rpc.so+0x2c228)
2021-03-26 19:47:58,215 (Log printer for 'ts2|pid9338|:30705|http://127.141.71.74:13264') [WARN - org.yb.minicluster.LogPrinter.runThread(LogPrinter.java:144)] Stream closed