test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x (LTO + pydebug) #109972

vstinner · 2023-09-27T14:42:59Z

PPC64LE RHEL8 3.x:

0:19:19 load avg: 1.37 [463/463/1] test_gdb process crashed (Exit code 1)
Timeout (0:15:00)!
Thread 0x00007fffae664d70 (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/selectors.py", line 398 in select
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 2108 in _communicate
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 1209 in communicate
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_gdb.py", line 107 in run_gdb
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_gdb.py", line 224 in get_stack_trace
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_gdb.py", line 949 in test_pycfunction
  (...)

(...)

0:19:19 load avg: 1.37 Re-running 1 failed tests in verbose mode in subprocesses
0:19:19 load avg: 1.37 Run 1 test in parallel using 1 worker process (timeout: 15 min, worker timeout: 20 min)
0:19:49 load avg: 1.22 running (1): test_gdb (30.0 sec)
0:20:19 load avg: 1.19 running (1): test_gdb (1 min)
0:20:49 load avg: 1.11 running (1): test_gdb (1 min 30 sec)
(...)
0:33:50 load avg: 1.09 running (1): test_gdb (14 min 30 sec)
0:34:20 load avg: 1.14 running (1): test_gdb (15 min)
0:34:20 load avg: 1.14 [1/1/1] test_gdb process crashed (Exit code 1)
Re-running test_gdb in verbose mode
test_NULL_ob_type (test.test_gdb.PrettyPrintTests.test_NULL_ob_type)
Ensure that a PyObject* with NULL ob_type is handled gracefully ... ok
(...)
test_pycfunction (test.test_gdb.PyBtTests.test_pycfunction)
Verify that "py-bt" displays invocations of PyCFunction instances ... Timeout (0:15:00)!
Thread 0x00007fffa14b4d70 (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/selectors.py", line 398 in select
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 2108 in _communicate
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 1209 in communicate
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_gdb.py", line 107 in run_gdb
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_gdb.py", line 224 in get_stack_trace
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_gdb.py", line 949 in test_pycfunction
  (...)

build: https://buildbot.python.org/all/#/builders/559/builds/3951

Linked PRs

The text was updated successfully, but these errors were encountered:

vstinner · 2023-09-27T14:50:07Z

Ah, it's a LTO build.

configure + make:

../configure --prefix '$(PWD)/target' --with-pydebug --with-platlibdir=lib64 --enable-ipv6 --enable-shared --with-computed-gotos=yes --with-dbmliborder=gdbm:ndbm:bdb --enable-loadable-sqlite-extensions --with-ssl-default-suites=openssl --without-static-libpython --with-lto
(...)
gcc -pthread -c -fno-strict-overflow -Wsign-compare -g -Og -Wall    -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I../Include/internal -IObjects -IInclude -IPython -I. -I../Include   -fPIC -DPy_BUILD_CORE -o Programs/python.o ../Programs/python.c
(...)
gcc -pthread   -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g  -Xlinker -export-dynamic -o python Programs/python.o -L. -lpython3.13d -lpthread -ldl  -lutil                        -lm

test.pythoninfo:

libregrtests.build_info: debug shared LTO

CC.version: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-18)

sysconfig[PY_CFLAGS]: -fno-strict-overflow -Wsign-compare -g -Og -Wall
sysconfig[PY_CFLAGS_NODIST]: -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I../Include/internal
sysconfig[PY_STDMODULE_CFLAGS]: -fno-strict-overflow -Wsign-compare -g -Og -Wall -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I../Include/internal -IObjects -IInclude -IPython -I. -I../Include -fPIC

sysconfig[PY_LDFLAGS_NODIST]: -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g
sysconfig[PY_CORE_LDFLAGS]: -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g

vstinner · 2023-09-27T14:51:32Z

On this buildbot, if Python is built with ./configure --with-pydebug and gcc -Og, test_gdb completes in 4 min 30 sec and looks reliable:

[vstinner@rh-power-vm13 cpython]$ ./python -m test test_gdb -u all -v
(...)
Total duration: 4 min 30 sec
Total tests: run=46 skipped=1
Total test files: run=1/1
Result: SUCCESS

vstinner · 2023-09-27T15:42:09Z

When Python is built with ./configure --with-pydebug --with-lto && make -j10, well, test_gdb is ... way slower: 19 min 45 sec.

test_gdb passed in 19 min 45 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 19 min 45 sec
Total tests: run=46 skipped=1
Total test files: run=1/1
Result: SUCCESS

vstinner · 2023-09-27T15:52:12Z

Python built with ./configure --with-pydebug CFLAGS="-O0" and gcc -O0:

test_gdb passed in 4 min 12 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 4 min 12 sec
Total tests: run=46 skipped=1
Total test files: run=1/1
Result: SUCCESS

vstinner · 2023-09-27T15:54:50Z

The test runs 130 gdb commands in total. Average:

gcc -O0: 1.9 sec / command
gcc -Og: 2.1 sec / command
gcc -Og -flto: 9.1 sec / command

vstinner · 2023-09-27T16:09:50Z

Number of gdb commands per test case:

PrettyPrintTests: 62

PyBtTests: 58

PyListTests: 6
PyLocalsTests: 5
PyPrintTests: 7
StackNavigationTests: 7

Split test_gdb.py file into a test_gdb package made 3 tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. * Split PyBtTests.test_pycfunction() into 6 functions: * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * In verbose mode, these "pycfunction" tests now log each tested call.

vstinner · 2023-09-27T16:46:00Z

I wrote PR #109977 to split test_gdb into a package of 3 tests so these 3 tests can be run in parallel.

Split test_gdb.py file into a test_gdb package made 3 tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. * Split PyBtTests.test_pycfunction() into 2 files (test_cfunction and test_cfunction_full) and 6 functions: * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * In verbose mode, these "pycfunction" tests now log each tested call. * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION.

Split test_gdb.py file into a test_gdb package made 3 tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. * Split PyBtTests.test_pycfunction() into 2 files (test_cfunction and test_cfunction_full) and 6 functions: * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * In verbose mode, these "pycfunction" tests now log each tested call. * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler. * Add check_gdb() function to util.py.

vstinner · 2023-09-28T09:08:34Z

The test runs 130 gdb commands in total. Average: gcc -Og -flto: 9.1 sec / command

Well, gdb takes 3.9 seconds just to load symbols from the Python ELF binary. Time measured with (before, I removed manually python-gdb.py):

time gdb -batch --init-eval-command "add-auto-load-safe-path $PWD" -args ./python

Split test_gdb.py file into a test_gdb package made 3 tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. * Split PyBtTests.test_pycfunction() into 2 files (test_cfunction and test_cfunction_full) and 6 functions: * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * In verbose mode, these "pycfunction" tests now log each tested call. * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler. * Add check_gdb() function to util.py. * Explicitly skip test_gdb on Windows.

Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select().

* Split PyBtTests.test_pycfunction() into 2 files, test_cfunction and test_cfunction_full, and 6 functions: * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * In verbose mode, these "pycfunction" tests now log each tested call. * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler. * Add check_gdb() function to util.py.

* Split PyBtTests.test_pycfunction() into 2 files, test_cfunction and test_cfunction_full, and 6 functions: * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * In verbose mode, these "pycfunction" tests now log each tested call. * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler. * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags.

* Split PyBtTests.test_pycfunction() into 2 files, test_cfunction and test_cfunction_full, and 6 functions: * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * In verbose mode, these "pycfunction" tests now log each tested call. * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler. * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files.

* Split test_pycfunction.py: add test_cfunction_full.py. Split the function into the following 6 functions. In verbose mode, these "pycfunction" tests now log each tested call. * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler instead of "replace". * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files.

Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select().

* Split test_pycfunction.py: add test_cfunction_full.py. Split the function into the following 6 functions. In verbose mode, these "pycfunction" tests now log each tested call. * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler instead of "replace". * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files.

Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select(). (cherry picked from commit 8f324b7)

…110339) gh-109972: Split test_gdb.py into test_gdb package (#109977) Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select(). (cherry picked from commit 8f324b7)

…on#109977) (python#110339) pythongh-109972: Split test_gdb.py into test_gdb package (python#109977) Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select(). (cherry picked from commit 8f324b7) (cherry picked from commit e7a61d3)

#110343) [3.12] gh-109972: Split test_gdb.py into test_gdb package (#109977) (#110339) gh-109972: Split test_gdb.py into test_gdb package (#109977) Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select(). (cherry picked from commit 8f324b7) (cherry picked from commit e7a61d3)

* Split test_pycfunction.py: add test_cfunction_full.py. Split the function into the following 6 functions. In verbose mode, these "pycfunction" tests now log each tested call. * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler instead of "replace". * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files. (cherry picked from commit 757cbd4)

* gh-109972: Enhance test_gdb (#110026) * Split test_pycfunction.py: add test_cfunction_full.py. Split the function into the following 6 functions. In verbose mode, these "pycfunction" tests now log each tested call. * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler instead of "replace". * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files. (cherry picked from commit 757cbd4) * gh-104736: Fix test_gdb tests on ppc64le with clang (#109360) Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex: Fedora 38). Search patterns in gdb "bt" command output to detect when gdb fails to retrieve the traceback. For example, skip a test if "Backtrace stopped: frame did not save the PC" is found. (cherry picked from commit 44d9a71) * gh-110166: Fix gdb CFunctionFullTests on ppc64le clang build (#110331) CFunctionFullTests now also runs "bt" command before "py-bt-full", similar to CFunctionTests which also runs "bt" command before "py-bt". So test_gdb can skip the test if patterns like "?? ()" are found in the gdb output. (cherry picked from commit 1de9406)

…110351) * pythongh-109972: Enhance test_gdb (pythonGH-110026) * Split test_pycfunction.py: add test_cfunction_full.py. Split the function into the following 6 functions. In verbose mode, these "pycfunction" tests now log each tested call. * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler instead of "replace". * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files. (cherry picked from commit 757cbd4) * pythongh-104736: Fix test_gdb tests on ppc64le with clang (pythonGH-109360) Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex: Fedora 38). Search patterns in gdb "bt" command output to detect when gdb fails to retrieve the traceback. For example, skip a test if "Backtrace stopped: frame did not save the PC" is found. (cherry picked from commit 44d9a71) * pythongh-110166: Fix gdb CFunctionFullTests on ppc64le clang build (pythonGH-110331) CFunctionFullTests now also runs "bt" command before "py-bt-full", similar to CFunctionTests which also runs "bt" command before "py-bt". So test_gdb can skip the test if patterns like "?? ()" are found in the gdb output. (cherry picked from commit bbce8bd) Co-authored-by: Victor Stinner <vstinner@python.org> (cherry picked from commit 1de9406)

…10354) [3.12] gh-109972: Enhance test_gdb (GH-110026) (GH-110351) * gh-109972: Enhance test_gdb (GH-110026) * Split test_pycfunction.py: add test_cfunction_full.py. Split the function into the following 6 functions. In verbose mode, these "pycfunction" tests now log each tested call. * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler instead of "replace". * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files. (cherry picked from commit 757cbd4) * gh-104736: Fix test_gdb tests on ppc64le with clang (GH-109360) Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex: Fedora 38). Search patterns in gdb "bt" command output to detect when gdb fails to retrieve the traceback. For example, skip a test if "Backtrace stopped: frame did not save the PC" is found. (cherry picked from commit 44d9a71) * gh-110166: Fix gdb CFunctionFullTests on ppc64le clang build (GH-110331) CFunctionFullTests now also runs "bt" command before "py-bt-full", similar to CFunctionTests which also runs "bt" command before "py-bt". So test_gdb can skip the test if patterns like "?? ()" are found in the gdb output. (cherry picked from commit bbce8bd) Co-authored-by: Victor Stinner <vstinner@python.org> (cherry picked from commit 1de9406) Co-authored-by: Victor Stinner <vstinner@python.org>

Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select().

* Split test_pycfunction.py: add test_cfunction_full.py. Split the function into the following 6 functions. In verbose mode, these "pycfunction" tests now log each tested call. * test_pycfunction_noargs() * test_pycfunction_o() * test_pycfunction_varargs() * test_pycfunction_varargs_keywords() * test_pycfunction_fastcall() * test_pycfunction_fastcall_keywords() * Move get_gdb_repr() to PrettyPrintTests. * Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT. * Rename checkout_hook_path to CHECKOUT_HOOK_PATH. * Rename gdb_version to GDB_VERSION_TEXT. * Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION. * run_gdb() uses "backslashreplace" error handler instead of "replace". * Add check_gdb() function to util.py. * Enhance support.check_cflags_pgo(): check also for sysconfig PGO_PROF_USE_FLAG (if available) in compiler flags. * Move some SkipTest checks to test_gdb/__init__.py. * Elaborate why gdb cannot be tested on Windows: gdb doesn't support PDB debug symbol files.

vstinner added the tests Tests in the Lib/test dir label Sep 27, 2023

vstinner changed the title ~~test_gdb failed with 15 min timeout on AMD64 RHEL8 Refleaks 3.x~~ test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x Sep 27, 2023

vstinner changed the title ~~test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x~~ test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x (LTO + pydebug) Sep 27, 2023

bedevere-app bot mentioned this issue Sep 27, 2023

gh-109972: Split test_gdb into a package of 3 tests #109977

Merged

bedevere-app bot mentioned this issue Sep 28, 2023

gh-109972: Enhance test_gdb #110026

Merged

vstinner closed this as completed Sep 28, 2023

bedevere-app bot mentioned this issue Oct 4, 2023

[3.12] gh-109972: Split test_gdb.py into test_gdb package (#109977) #110339

Merged

vstinner mentioned this issue Oct 4, 2023

[3.11] gh-109972: Split test_gdb.py into test_gdb package (#109977) (… #110343

Merged

bedevere-app bot mentioned this issue Oct 4, 2023

[3.12] gh-109972: Enhance test_gdb (#110026) #110351

Merged

miss-islington mentioned this issue Oct 4, 2023

[3.11] [3.12] gh-109972: Enhance test_gdb (GH-110026) (GH-110351) #110354

Merged

vstinner mentioned this issue Jan 18, 2024

PPC64LE RHEL8 3.x buildbot is timing out on test_gdb #104735

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x (LTO + pydebug) #109972

test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x (LTO + pydebug) #109972

vstinner commented Sep 27, 2023 •

edited by bedevere-app bot

Loading

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 28, 2023

test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x (LTO + pydebug) #109972

test_gdb failed with 15 min timeout on PPC64LE RHEL8 3.x (LTO + pydebug) #109972

Comments

vstinner commented Sep 27, 2023 • edited by bedevere-app bot Loading

Linked PRs

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 27, 2023

vstinner commented Sep 28, 2023

vstinner commented Sep 27, 2023 •

edited by bedevere-app bot

Loading