Segmentation Fault on Python application exit with Python 3.9.5

[ygorelik]

Current Behavior

When upgraded Python to version 3.9.5, it is noticed that all YDK Python applications exit with Segmentation Fault.

Investigation showed that this is well known Python 3.9 issue documented in multiple resources; here is one of them.

Root cause: At application exit the pybind objects got deleted AFTER corresponding memory was released.

Steps to Reproduce

After Python upgrade/installation to version 3.9.x and YDK installation run sanity test and notice segmentation fault:

python -c "import ydk.types"

Logs

The following application crash information was obtained on MACOS Mojave with Python 3.5.9:

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000010
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [29155]

VM Regions Near 0x10:
--> 
    __TEXT      0000000108b7c000-0000000108b7e000 [    8K] r-x/r-x SM=COW 
/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   org.python.python             0x0000000108beeb06 method_dealloc + 102
1   ydk_.so                                0x0000000109a21156 pybind11::object::~object() + 22
2   libsystem_c.dylib               0x00007fff5cb7c3cf __cxa_finalize_ranges + 319
3   libsystem_c.dylib               0x00007fff5cb7c6b3 exit + 55
4   org.python.python             0x0000000108d34f7f Py_Exit + 30
5   org.python.python             0x0000000108d3895e handle_system_exit + 35
6   org.python.python             0x0000000108d38363 _PyErr_PrintEx + 42
7   org.python.python             0x0000000108d37b00 pyrun_simple_file + 439
8   org.python.python             0x0000000108d37923 PyRun_SimpleFileExFlags + 67
9   org.python.python             0x0000000108d565dd pymain_run_file + 327
10  org.python.python            0x0000000108d55d2d pymain_run_python + 460
11  org.python.python             0x0000000108d55b25 Py_RunMain + 23
12  org.python.python             0x0000000108d56e42 pymain_main + 35
13  org.python.python             0x0000000108d570c4 Py_BytesMain + 42
14  libdyld.dylib                        0x00007fff5cad63d5 start + 1

System Information

YDK installation on MACOS Mohave with Python 3.9.5.

Workaround

Downgrade Python to version 3.7.x, which is proven to be stable for YDK applications.

[ygorelik]

This issue persist with Python 3.10.4.

[ygorelik]

Related to pybind11 documented issue 3108.

[ekinzie]

I doubt this is a real fix, but I've worked around this problem with the following:

index 135fe85b..a4f42608 100644
--- a/sdk/python/core/python.cpp
+++ b/sdk/python/core/python.cpp
@@ -92,10 +92,15 @@ void setup_logging()

         add_null_handler(logger);
         log_debug = logger.attr("debug");
+        log_debug.inc_ref();
         log_info = logger.attr("info");
+        log_info.inc_ref();
         log_warning = logger.attr("warning");
+        log_warning.inc_ref();
         log_error = logger.attr("error");
+        log_error.inc_ref();
         log_critical = logger.attr("critical");
+        log_critical.inc_ref();

         ydk::set_logging_callback("debug", debug);
         ydk::set_logging_callback("info", info);
diff --git a/sdk/python/gnmi/python_gnmi.cpp b/sdk/python/gnmi/python_gnmi.cpp
index 7569e374..a424fd70 100644
--- a/sdk/python/gnmi/python_gnmi.cpp
+++ b/sdk/python/gnmi/python_gnmi.cpp
@@ -82,10 +82,15 @@ static void setup_gnmi_logging()

         add_null_handler(logger);
         log_debug = logger.attr("debug");
+        log_debug.inc_ref();
         log_info = logger.attr("info");
+        log_info.inc_ref();
         log_warning = logger.attr("warning");
+        log_warning.inc_ref();
         log_error = logger.attr("error");
+        log_error.inc_ref();
         log_critical = logger.attr("critical");
+        log_critical.inc_ref();

[ygorelik]

This is great, ekinzie! Thank you for the contribution. I will include your fix to release 0.8.6.4.

It is quite possible that it also fixes the issue #1074, which is related to the logging.

[ygorelik]

Applied this fix to maintenance release 0.8.6.5 (under development).