Preview: point release v23.1.0

[pajod]

This is where I put together various PRs.

The following PRs merge OK:

• Add socket backlog metric benoitc/gunicorn#2407
• Fix reuse-port to balance requests across Gunicorn workers benoitc/gunicorn#2938 (see script below)
• CI: Pylint 3.x benoitc/gunicorn#3110
• arbiter: Handle SIGCHLD like all other signals + misc signal handling improvements benoitc/gunicorn#3148
• workers/gthread: Remove locks + one event queue + general cleanup benoitc/gunicorn#3157
• Python 3.4+: use os.set_inheritable + os.set_blocking in place of unreadable fcntl calls benoitc/gunicorn#3210
• Close open resources benoitc/gunicorn#3250
• assert only, no change intended: never reflect invalid input benoitc/gunicorn#3264
• config: permit --reload-extra-files without --reload benoitc/gunicorn#3271
• style: argparse inheritance, hide some options from --help benoitc/gunicorn#3272
• style: re-verbosify HTTP commentary benoitc/gunicorn#3273
• revert: 4023228 ("let's exception not bubble") benoitc/gunicorn#3275

These merge conflicts are straightforward to resolve:

• CI: Pylint 3.x benoitc/gunicorn#3110

I recommend most of the above go into a 23.1.0 bugfix release.

These patches are included to help testing here, albeit not quite ready:

• Add options to purge big workers benoitc/gunicorn#3252
• https://github.com/pajod/gunicorn/tree/docs-misc (no PR, unsubmitted)

Please do not actually octopus merge though, sequential merges produce clearer output in common git tools. Created using this script:

#!/bin/sh
# assuming .git/config fetches pull requests from GitHub, e.g:
# [remote "benoitc"]
#         url = https://github.com/benoitc/gunicorn.git
#         fetch = +refs/heads/*:refs/remotes/benoitc/*
#         fetch = +refs/pull/*/head:refs/remotes/benoitc/pr/*
# [remote "pajod"]
#         url = https://github.com/pajod/gunicorn.git
#         fetch = +refs/heads/*:refs/remotes/pajod/*
set -e
echo "preparing branch"
test -z "$(git status --porcelain=v1 --untracked-files=no)" || exit 1
git fetch benoitc
git checkout -B integration-v23.1.0 benoitc/master
git reset --hard benoitc/master
git merge --verbose --no-edit --no-rerere-autoupdate benoitc/pr/2938
git apply -3 <<'EOF'
diff --git a/gunicorn/arbiter.py b/gunicorn/arbiter.py
index 646d684e..4d8a1b3b 100644
--- a/gunicorn/arbiter.py
+++ b/gunicorn/arbiter.py
@@ -7,8 +7,8 @@ import random
 import select
 import signal
+import socket
 import sys
 import time
 import traceback
-import socket

 from gunicorn.errors import HaltServer, AppImportError
EOF
git commit -m "help merge imports" -- gunicorn/arbiter.py
echo "octopus merge"
git merge --verbose --no-edit --no-rerere-autoupdate benoitc/pr/2407 \
       benoitc/pr/3157 benoitc/pr/3273 benoitc/pr/3271 \
       benoitc/pr/3275 benoitc/pr/3148 benoitc/pr/3250 \
       benoitc/pr/3264 benoitc/pr/3210 benoitc/pr/3272 \
       pajod/docs-misc
echo "docs"
git apply -3 <<'EOF'
diff --git a/tests/test_nginx.py b/tests/test_nginx.py
index 69742edb..07e07239 100644
--- a/tests/test_nginx.py
+++ b/tests/test_nginx.py
@@ -452,3 +452,3 @@ def test_nginx_proxy(*, ssl, worker_class, dummy_ssl_cert, read_size=1024):
                 expect={
-                    "Handling signal: term",
+                    "Handling signal: SIGTERM",
                     "Shutting down: Master",
EOF
git apply -3 <<'EOF'
diff --git a/tests/test_wrk.py b/tests/test_wrk.py
index f89dc013..fdce18a1 100644
--- a/tests/test_wrk.py
+++ b/tests/test_wrk.py
@@ -403,3 +403,3 @@ def test_wrk(*, ssl, worker_class, dummy_ssl_cert, read_size=1024):
                 expect={
-                    "[INFO] Handling signal: term",
+                    "[INFO] Handling signal: SIGTERM",
                 },
EOF
git commit -m "help merge test for modified logging" -- tests/test_nginx.py tests/test_wrk.py
(cd docs && sphinx-build -n -b html -d build/doctrees source build/html )
git commit --allow-empty -m "prepare docs 23.1.0" -- docs/source/settings.rst
echo "version commit"
sed -i -e 's/23, 0, 0/23, 1, 0/' gunicorn/__init__.py
git commit -m "bump version 23.1.0" -- gunicorn/__init__.py
echo "checking"
pylint -j0 --ignore-paths="^tests/requests/" --max-line-length=120 gunicorn/ tests/ || true
pycodestyle --exclude="tests/requests/*" gunicorn tests || true
python -B -m pytest -s -vvvv --ff --override-ini=addopts=--strict-markers --exitfirst || true
echo "building"
pip --quiet install -U build
python -m build
echo "searching for expected file in sdist"
tar -tf dist/gunicorn-23.1.0.tar.gz  | grep -m 1 gunicorn/__main__.py || true
echo "searching for unexpected file in sdist"
tar -tf dist/gunicorn-23.1.0.tar.gz  | grep -m 1 docs/build || echo 'none found (good!)'
echo "done"

[sylt]

@pajod I've updated/synced my two PRs now, so hopefully they should (if they are accepted) merge cleanly!

[sylt]

I tested your integration branch a little bit, and I found a bug in my code where I tested the scenario of sending SIGHUP to the arbiter while running a gthread type worker. The following hunk needs to be applied:

diff --git a/gunicorn/workers/gthread.py b/gunicorn/workers/gthread.py
index 196759b8..b47ddaef 100644
--- a/gunicorn/workers/gthread.py
+++ b/gunicorn/workers/gthread.py
@@ -118,8 +118,9 @@ class ThreadWorker(base.Worker):
         return futures.ThreadPoolExecutor(max_workers=self.cfg.threads)
 
     def handle_exit(self, sig, frame):
-        self.alive = False
-        self.method_queue.defer(lambda: None)  # To wake up poller.select()
+        if self.alive:
+            self.alive = False
+            self.method_queue.defer(lambda: None)  # To wake up poller.select()
 
     def handle_quit(self, sig, frame):
         self.thread_pool.shutdown(False)

With this in place, I can spam SIGHUP to the arbiter while running ab in parallel without losing any requests.

I will update my pull request later today!

[sylt]

Now it's fixed! Sorry for any trouble.