-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
API to parse requirements.txt for setup.py #1080
Comments
There's also no support for environment markers in Note that with open('requirements.txt') as fp:
install_requires = fp.read()
setup(install_requires=install_requires, ...) in your Also looks like a duplicate of #1074. |
Nice. Save me a lot of time. If only I could detect when
Are those differences documented somewhere? |
Well, at the very least, setuptools should error out if environment markers are used: setuptools/dist.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git i/setuptools/dist.py w/setuptools/dist.py
index 6b97ed33..eb9e4442 100644
--- i/setuptools/dist.py
+++ w/setuptools/dist.py
@@ -153,7 +153,11 @@ def assert_bool(dist, attr, value):
def check_requirements(dist, attr, value):
"""Verify that install_requires is a valid requirements list"""
try:
- list(pkg_resources.parse_requirements(value))
+ reqs = []
+ for r in pkg_resources.parse_requirements(value):
+ if r.marker:
+ raise ValueError("environment markers are not supported, in '%s'" % r)
+ reqs.append(r)
except (TypeError, ValueError) as error:
tmpl = (
"{attr!r} must be a string or list of strings " And the documentation should be fixed, as both direct install with |
After inspection, environment markers do work when installing from source, but not with the generated wheels, see #1081. |
Agreed this is a duplicate of #1074. |
Might be a stupid question, but does pip's
From like:
? |
#1074 needs a better title. Also, there are two scenarios in parsing
2 is actually two step process, which includes step 1. But because 2 is more often needed than 1, it is mixed up together and makes API hard. |
@tuukkamustonen https://stackoverflow.com/a/43090648/239247 - is that it? |
@techtonik That is for depending on named extras in other packages. I'm after being able to declare them in mine, so to say something like So some format that I can parse from
|
@tuukkamustonen so, you want a mechanism to specify different sets of optional dependencies? I believe it is done with tags specified in square brackets.
https://www.python.org/dev/peps/pep-0508/#extras |
@techtonik That syntax just gives a parsing error (in pip 9.0.1). |
@tuukkamustonen then it is a bug. |
@tuukkamustonen You use setup.py to declare your package's dependencies (and extras). You use requirements.txt to install any number of packages (and their dependencies). So if you want to declare an extra scope for your package, you do that in setup.py:
Then, if you want to install your package with the extra scope, you can Does that help? |
@jaraco Thanks for the write-up, but my question was if I can do that ( The reason I asked this in the context of this ticket is that if I wanted to parse named extras from Maybe an easier to understand use case would be this: Let's forget Why would I split the dev requirements into several files? In CI (e.g. Jenkins), not every job needs all the dependencies, and maybe I want to optimize the time it takes to setup a virtualenv (let's say I'm spinning up a new Docker container for each build). In order to avoid cluttering my repository with
And then being able to run something like:
I followed the syntax suggested by #1080 (comment) here (but note that it gives a parsing error). According to @techtonik that's a bug. There may be some confusion here, so is it really a bug, or just feature that is not specified/supported? |
To include all the testing dependencies required by pytest-runner, but that's not a way to declare an optional |
So that's just not supported then. Ok. Thanks for clarifying things up. |
requirements.txt supports this, setup.py does not: setup.py gives an error like this: stuff like this this is still a valid question and a confusing issue years and years later. this api should exist and maybe parse requirements.txt to return a dictionary of |
This can cause problems with hyphenated package names. It is fairly common for PyPi packages to be listed with a hyphen in their name, but for all other references to them to have underscores. An example package is pytorch-transformers, where you Because of this, you need to replace import requirements
install_requires = []
with open('requirements.txt') as fd:
for req in requirements.parse(fd):
if req.name:
name = req.name.replace("-", "_")
full_line = name + "".join(["".join(list(spec)) for spec in req.specs])
install_requires.append(full_line)
else:
# a GitHub URL doesn't have a name
# I am not sure what to do with this, actually, right now I just ignore it...
setup(install_requires=install_requires, ...) It's not pretty, and it adds a dependency, though. |
Judging from madpah/requirements-parser#45 the author probably would not object if |
requirements.txt
could a convenient single declarative point for specifying dependencies if it could be pasted intosetup.py
install_requires
as-is. Butrequirements.txt
contains comments and references to other requirements files, which seem to be incompatible withinstall_requires
(reference needed).It would be nice to have official API that parses
requirements.txt
file into a list suitable forinstall_requires
section.https://stackoverflow.com/questions/14399534/reference-requirements-txt-for-the-install-requires-kwarg-in-setuptools-setup-py
The text was updated successfully, but these errors were encountered: