Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-94518: [_posixsubprocess] Replace variable validity flags with reserved values #94687

Merged
merged 14 commits into from
Jan 14, 2023

Conversation

arhadthedev
Copy link
Member

@arhadthedev arhadthedev commented Jul 8, 2022

Currently, _posixsubprocess.c uses group and user ids together with flags of whether the corrsponding variables were initialized. However such an implicit "either initialized or look somewhere else" confuses both a reader (another mental connection to constantly track between functions) and a compiler (warnings on potentially uninitialized variables). Instead, we can utilize a special group/user id as a flag value defined by POSIX but uses nowhere else. Namely:

  • gid: call_setgid = Falsegid = -1
  • uid: call_setuid = Falseuid = -1
  • groups: call_setgroups = Falsegroups = NULL (obtained with (groups_list != Py_None) ? groups : NULL)

This PR is required for gh-94519.


PS.: @tiran I applied your suggestion from gh-94519 as this, separate PR to keep that one limited in scope and reviewable.

@arhadthedev arhadthedev requested a review from gpshead as a code owner July 8, 2022 11:31
@arhadthedev arhadthedev changed the title gh-94518: [_posixsubprocess] Replace each variable validity flag with a reserved value of that variable gh-94518: [_posixsubprocess] Replace variable validity flags with reserved values Jul 8, 2022
@arhadthedev
Copy link
Member Author

This change has no user-visible effect so no NEWS entry is required.

@arhadthedev
Copy link
Member Author

Please include a news entry.

We typically only skip news if the change is a trivial internal change or change is already covered by a previous PR. Non-trivial commits get a news entry so other core devs, release manager, distributors, and users have an easier way to locate a problem when a change introduces a regression.

@tiran Done (initially I've thought this change is invisible to users so skipped it).

Copy link
Member

@gpshead gpshead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this PR :)

Modules/_posixsubprocess.c Outdated Show resolved Hide resolved
Modules/_posixsubprocess.c Outdated Show resolved Hide resolved
Modules/_posixsubprocess.c Outdated Show resolved Hide resolved
Modules/_posixsubprocess.c Outdated Show resolved Hide resolved
Modules/_posixsubprocess.c Show resolved Hide resolved
Modules/_posixsubprocess.c Outdated Show resolved Hide resolved
@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@gpshead gpshead self-assigned this Jul 27, 2022
When we query `groups_list` size using `PySequence_Size()`, we get -1
for non-lists. It returns an ability to pass an empty list to
`setgroups()` that is valid but treated as a no-op:

> If `size` is zero, `list` is not modified, but the total number of
> supplementary group IDs for the process is returned. This allows the
> caller to determine the size of a dynamically allocated list to be
> used in a further call to getgroups().
>
> - <https://linux.die.net/man/2/setgroups>
`num_groups < 0` already implies `groups_list == Py_None` because
`num_groups = PySequence_Size(groups_list)` does this check for us.

Also, it guarantees that `groups_list == Py_None` <=> `groups == NULL`;
it allows to replace `groups_list != Py_None ? groups : NULL` with just
`groups` in a call of `do_fork_exec()`.
@arhadthedev
Copy link
Member Author

arhadthedev commented Jul 27, 2022

if (groups_size >= 0)

setgroups(0, NULL) is valid.

For some reason, setgroups(0, NULL) causes Doctest, Check if generated files are up to date, and SSL workflows to fail child processes with PermissionError: [Errno 1] Operation not permitted exception:

Stacktrace
Run ./python Lib/test/ssltests.py
Traceback (most recent call last):
OpenSSL 1.1.1n  15 Mar 2022
  File "/home/runner/work/cpython/cpython/Lib/test/ssltests.py", line 37, in <module>
    run_regrtests(*sys.argv[1:])
  File "/home/runner/work/cpython/cpython/Lib/test/ssltests.py", line 33, in run_regrtests
    result = subprocess.call(args)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/Lib/subprocess.py", line 378, in call
    with Popen(*popenargs, **kwargs) as p:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/Lib/subprocess.py", line 1011, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/runner/work/cpython/cpython/Lib/subprocess.py", line 1888, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 1] Operation not permitted
Error: Process completed with exit code 1.

However, this is fixed by excluding a zero from the comparison: if (groups_size > 0). I think we may do it this way because setgroups(0, NULL) is a no-op in child_exec (a positive returned value is dropped anyway):

If size is zero, list is not modified, but the total number of supplementary group IDs for the process is returned.

-- https://linux.die.net/man/2/setgroups

For Bedevere: I have made the requested changes; please review again. For correct jump from Mentioned to this message instead of the following Bedevere's one: @gpshead.

@bedevere-bot
Copy link

Thanks for making the requested changes!

@gpshead: please review the changes made to this pull request.

@arhadthedev
Copy link
Member Author

@gpshead @tiran could you reconsider this PR please? This is a prerequirement for gh-94519, created to get rid of uninitialized variables.

@tiran
Copy link
Member

tiran commented Aug 18, 2022

We are currently busy with 3.11 release. It may take until 3.11.0 final before we come back to this patch. It's definitely on the list for 3.12.

@arhadthedev
Copy link
Member Author

Todo: replace groups_size/groups parameter names with extra_group_size/extra_groups across the whole file to avoid confusion of passers-by. It's necessary to mitigate a misleading name of setgroups that supposes replacement of not supplimentary group affinities, but all ones. The mislead results in seeing the omission of setgroups call for groups_size == 0 as a bug (like we should reset the inherited affinities but do nothing instead).

I'll do it in a separate PR because both this PR and gh-94519 are already big enough to clobber them with extra details.

@arhadthedev
Copy link
Member Author

@tiran could you review and possibly merge this PR please? It's an implementation of your proposal #94519 (comment) created as a separate PR to preserve diffs of both PRs readable.

Copy link
Member

@gpshead gpshead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for doing this!

@arhadthedev
Copy link
Member Author

Thank you for merging, Gregory!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants