Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle virtual / phony make targets consistently and more robustly #104490

Closed
indygreg opened this issue May 15, 2023 · 1 comment · Fixed by #104491
Closed

Handle virtual / phony make targets consistently and more robustly #104490

indygreg opened this issue May 15, 2023 · 1 comment · Fixed by #104491
Labels
type-feature A feature request or enhancement

Comments

@indygreg
Copy link
Contributor

indygreg commented May 15, 2023 •
edited by bedevere-bot
Loading

Feature or enhancement

I already typed a detailed commit message explaining this. Will be explained by forthcoming PR linking to this issue.

Linked PRs
@indygreg indygreg added the type-feature A feature request or enhancement label May 15, 2023
@bedevere-bot bedevere-bot mentioned this issue May 15, 2023
Merged
indygreg added a commit to indygreg/cpython that referenced this issue May 15, 2023
@indygreg
pythongh-104490: consistently define phony make targets
aafd614 
By convention make targets that don't refer to a file have a dependency
on the fake `.PHONY` target/file. This ensures that these targets are
always evaluated because there is no rule to create a `.PHONY` file
and that will force make to think the rule is out of date and needs
to be rebuilt.

This commit consistently associates our virtual targets with `.PHONY`
by declaring the `.PHONY` dependency immediately above the make rule.

Before, some `.PHONY` associations were inline (as this patch
establishes). Others were defined in common blocks. Even others were
defined incorrectly, actually assigning a `.PHONY` variable instead
of operating on the make target!

Before, not all virtual targets had `.PHONY` associations. And there
were some defined `.PHONY` associations in common blocks whose targets
had long been deleted.

Hopefully the unified convention will prevent things from getting out
of sync going forward.

Why do this?

First, consistency is good. Now people don't have to know to look for
a separate block of `.PHONY` attribution. Hopefully this prevents
missing `.PHONY` annotations going forward.

Second, there are some cases where virtual make targets are virtual
/ phony when they shouldn't be. These are creating race conditions and
avoidable rebuilds across multiple `make` invocations. e.g. a `make all`
+ `make install` invocation will perform PGO+BOLT if BOLT is enabled
because `profile-opt` and `bolt-opt` are phony and always invoked. I
plan follow-ups to this cleanup to fix some bugs I've run into.
@erlend-aasland
Copy link
Contributor

The reason and rationale for the change is given in the PR (gh-104491). I'm quoting it here:

By convention make targets that don't refer to a file have a dependency on the fake .PHONY target/file. This ensures that these targets are always evaluated because there is no rule to create a .PHONY file and that will force make to think the rule is out of date and needs to be rebuilt.

This commit consistently associates our virtual targets with .PHONY by declaring the .PHONY dependency immediately above the make rule.

Before, some .PHONY associations were inline (as this patch establishes). Others were defined in common blocks. Even others were defined incorrectly, actually assigning a .PHONY variable instead of operating on the make target!

Before, not all virtual targets had .PHONY associations. And there were some defined .PHONY associations in common blocks whose targets had long been deleted.

Hopefully the unified convention will prevent things from getting out of sync going forward.

Why do this?

First, consistency is good. Now people don't have to know to look for a separate block of .PHONY attribution. Hopefully this prevents missing .PHONY annotations going forward.

Second, there are some cases where virtual make targets are virtual / phony when they shouldn't be. These are creating race conditions and avoidable rebuilds across multiple make invocations. e.g. a make all

  • make install invocation will perform PGO+BOLT if BOLT is enabled because profile-opt and bolt-opt are phony and always invoked. I plan follow-ups to this cleanup to fix some bugs I've run into.

@erlend-aasland erlend-aasland linked a pull request May 15, 2023 that will close this issue
gh-104490: Consistently define phony make targets #104491
Merged
erlend-aasland pushed a commit that referenced this issue May 15, 2023
@indygreg
gh-104490: Consistently define phony make targets (#104491)
a6bcc8f 
By convention make targets that don't refer to a file have a dependency
on the fake .PHONY target/file. This ensures that these targets are
always evaluated because there is no rule to create a .PHONY file
and that will force make to think the rule is out of date and needs
to be rebuilt.

This commit consistently associates virtual targets with .PHONY by
declaring the .PHONY dependency immediately above the make rule. This
should avoid race conditions and avoidable rebuilds across multiple make
invocations.
carljm added a commit to carljm/cpython that referenced this issue May 15, 2023
@carljm
Merge branch 'main' into supermon
343867a 
* main: (29 commits)
  pythongh-101819: Fix _io clinic input for unused base class method stubs (python#104418)
  pythongh-101819: Isolate `_io` (python#101948)
  Bump mypy from 1.2.0 to 1.3.0 in /Tools/clinic (python#104501)
  pythongh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (python#104495)
  pythongh-104050: Run mypy on `clinic.py` in CI (python#104421)
  pythongh-104490: Consistently define phony make targets (python#104491)
  pythongh-67056: document that registering/unregistering an atexit func from within an atexit func is undefined (python#104473)
  pythongh-104487: PYTHON_FOR_REGEN must be minimum Python 3.10 (python#104488)
  pythongh-101282: move BOLT config after PGO (pythongh-104493)
  pythongh-104469 Convert _testcapi/float.c to use AC (pythongh-104470)
  pythongh-104456: Fix ref leak in _ctypes.COMError (python#104457)
  pythongh-98539: Make _SSLTransportProtocol.abort() safe to call when closed (python#104474)
  pythongh-104337: Clarify random.gammavariate doc entry  (python#104410)
  Minor improvements to typing docs (python#104465)
  pythongh-87092: avoid gcc warning on uninitialized struct field in assemble.c (python#104460)
  pythonGH-71383: IDLE - Document testing subsets of modules (python#104463)
  pythongh-104454: Fix refleak in AttributeError_reduce (python#104455)
  pythongh-75710: IDLE - add docstrings and comments to editor module (python#104446)
  pythongh-91896: Revert some very noisy DeprecationWarnings for `ByteString` (python#104424)
  Add a mention of PYTHONBREAKPOINT to breakpoint() docs (python#104430)
  ...
carljm added a commit to carljm/cpython that referenced this issue May 15, 2023
@carljm
Merge branch 'main' into cached_property_docs
452d9f5 
* main: (204 commits)
  pythongh-101819: Fix _io clinic input for unused base class method stubs (python#104418)
  pythongh-101819: Isolate `_io` (python#101948)
  Bump mypy from 1.2.0 to 1.3.0 in /Tools/clinic (python#104501)
  pythongh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (python#104495)
  pythongh-104050: Run mypy on `clinic.py` in CI (python#104421)
  pythongh-104490: Consistently define phony make targets (python#104491)
  pythongh-67056: document that registering/unregistering an atexit func from within an atexit func is undefined (python#104473)
  pythongh-104487: PYTHON_FOR_REGEN must be minimum Python 3.10 (python#104488)
  pythongh-101282: move BOLT config after PGO (pythongh-104493)
  pythongh-104469 Convert _testcapi/float.c to use AC (pythongh-104470)
  pythongh-104456: Fix ref leak in _ctypes.COMError (python#104457)
  pythongh-98539: Make _SSLTransportProtocol.abort() safe to call when closed (python#104474)
  pythongh-104337: Clarify random.gammavariate doc entry  (python#104410)
  Minor improvements to typing docs (python#104465)
  pythongh-87092: avoid gcc warning on uninitialized struct field in assemble.c (python#104460)
  pythonGH-71383: IDLE - Document testing subsets of modules (python#104463)
  pythongh-104454: Fix refleak in AttributeError_reduce (python#104455)
  pythongh-75710: IDLE - add docstrings and comments to editor module (python#104446)
  pythongh-91896: Revert some very noisy DeprecationWarnings for `ByteString` (python#104424)
  Add a mention of PYTHONBREAKPOINT to breakpoint() docs (python#104430)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

gh-104490: Consistently define phony make targets indygreg/cpython
2 participants
@indygreg @erlend-aasland