poetry>=1.2 package with group dependencies cannot be installed with default build-system

[lindycoder]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Monterey 12.0.1
• Poetry version: 1.2.0a2
• pyproject.toml: default from poetry init -n

Issue

Hello! Thanks a lot for this tool.

When creating a project with poetry>=1.2 it adds the default build-system:

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

If you add a --group dependency in this project, it becomes impossible to install because the latest stable release of poetry-core does not support group and produces

Traceback (most recent call last):
  File "/tmp/tmpbqlvtksr_in_process.py", line 280, in <module>
    main()
  File "/tmp/tmpbqlvtksr_in_process.py", line 263, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/tmp/tmpbqlvtksr_in_process.py", line 133, in prepare_metadata_for_build_wheel
    return hook(metadata_directory, config_settings)
  File "/tmp/pip-build-env-l3emamb6/overlay/lib/python3.10/site-packages/poetry/core/masonry/api.py", line 43, in prepare_metadata_for_build_wheel
    poetry = Factory().create_poetry(Path(".").resolve(), with_dev=False)
  File "/tmp/pip-build-env-l3emamb6/overlay/lib/python3.10/site-packages/poetry/core/factory.py", line 43, in create_poetry
    raise RuntimeError("The Poetry configuration is invalid:\n" + message)
RuntimeError: The Poetry configuration is invalid:
  - Additional properties are not allowed ('group' was unexpected)

Reproduce

This Dockerfile shows the issue

FROM python

RUN pip install "poetry==1.2.0a2"
# Create a package with a dev dependency
RUN mkdir -p /mypkg/mypkg \
    && cd /mypkg \
    && touch mypkg/__init__.py \
    && touch README.md \ 
    && poetry init -n \
    && poetry add --group dev pytest
 
 # Create a project that will import the package
 RUN mkdir -p /importer \
    && cd /importer \
    && poetry init -n
 
 # Trigger the error
 RUN cd /importer && poetry add /mypkg

Mitigation

Changing the build dependency to requires = ["poetry-core>=1.1.0a6"] will currently fix the issue

Resolution

• If poetry 1.1 was forward compatible (as raised here poetry 1.1 and forward compatibility with pyproject.toml using groups #4628) it could "maybe work" if the only groups are dev dependencies not meant to be installed with the package.
• Maybe poetry 1.2 could set a minimum build dependency, which would be poetry-core>=1.1 but i did not find a way to tell the build system to install with --pre to allow for pre-releases, so it would need to be explicitly something like poetry-core>=1.1.0a6.

Maybe this is not an issue, once poetry==1.2 and poetry-core==1.1 are officially release this might all go away.

Thank you very much!

[kakulukia]

I was just stumbling upon this while trying to build pendulum. Is there a way around this?

[lindycoder]

@kakulukia From the look of pendulum's pyproject.toml they use poetry >=1.2. Do you have this version installed? Needs to be installed using the preview flag, see installation here : https://python-poetry.org/docs/master/

[kakulukia]

I was not aware that its using a preview version of poetry .. thx, its working now.

[ljnsn]

I have the same issue on version 1.2.0b1.

[AkechiShiro]

Hey, why is pendulum requiring a preview version of poetry for installation was this necessary ? I had to build from source poetry as the one I use from my classical repos is too outdated, but I find this weird, for one package to be installed, we have to update the whole package manager...

[Secrus]

Hey, why is pendulum requiring a preview version of poetry for installation was this necessary ? I had to build from source poetry as the one I use from my classical repos is too outdated, but I find this weird, for one package to be installed, we have to update the whole package manager...

I don't think this is an issue of Poetry, but a decision of Pendulum maintainers. You should raise the question in the Pendulum repository.

[AkechiShiro]

Hey @Secrus, just for reference, it seems it isn't a Pendulum issue but a pip issue : pypa/pip#6264

[TBBle]

To be clear, the original issue this ticket was raised for still exists: A Poetry 1.2-managed project which uses groups (per the original example) still declares an unversioned build-system requirement on poetry-core, and so will fail if this is not manually adjusted to depend on poetry-core>=1.1.0a6 or similar. I just checked this hasn't changed in Poetry master (from earlier this week, anyway).

Pendulum already has this versioned build-system requirement, so it's not suffering from the issue in this ticket.

Poetry itself (in master) depends on poetry-core ^1.1.0b3, so it might make sense for Poetry projects created with poetry 1.2 to depend on that version of poetry-core for their build-system from the start; for existing projects, that requirement could be adjusted when some poetry-core-1.0-incompatible change , e.g. groups, or the newer console scripts support, is made via poetry CLI.

It might also make sense for the documentation to mark those pyproject.toml features that depend on poetry-core 1.1 in the documentation so that people editing their pyproject.toml manually with reference to the documentation, e.g., as happened in #4628, will know they need to ensure their build-system dependency is appropriately updated.

And as a further molly-guard, it might make sense to have Poetry notice when pyproject.toml contains features needing poetry-core 1.1 but does not declare this in its build-system block, similar to how it notices when your poetry.lock is out-of-date compared to pyproject.toml.

All this depends on having a clear record and documentation of the poetry-core 1.1 added and changed features, and the ability to recognise them in pyproject.toml. And parse the build-system requirement line to determine which version of poetry-core is being depended upon..

Separately, it might make sense for Poetry 1.2, when given add --group dev, to generate the poetry-core 1.0 format if the build-system requirement is not versioned for poetry-core 1.1 already (as opposed to bumping the poetry-core dependency as I suggested above). However, I assume that option would have been discussed in the initial implementation of generalised group support and not seen as valuable at the time.

[finswimmer]

The original issue is solved, because poetry-core 1.1 is out.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.