Remove unused dependencies, add deptry to make sure the dependencies stay up-to-date

[fpgmaas]

Running deptry on the codebase with this configuration

[tool.deptry]
extend_exclude = ["examples", "benchmarks", "outlines/integrations", "outlines/models"]
pep621_dev_dependency_groups = ["test"]

gives:

outlines/fsm/types.py:5:1: DEP003 'typing_extensions' imported but it is a transitive dependency
outlines/generate/api.py:161:16: DEP004 'torch' imported but declared as a dev dependency
outlines/generate/api.py:279:16: DEP004 'torch' imported but declared as a dev dependency
outlines/generate/generator.py:65:12: DEP004 'torch' imported but declared as a dev dependency
outlines/generate/generator.py:205:12: DEP004 'torch' imported but declared as a dev dependency
outlines/generate/generator.py:228:12: DEP004 'torch' imported but declared as a dev dependency
outlines/generate/generator.py:269:12: DEP004 'torch' imported but declared as a dev dependency
outlines/generate/generator.py:304:12: DEP004 'torch' imported but declared as a dev dependency
outlines/samplers.py:67:16: DEP004 'torch' imported but declared as a dev dependency
outlines/samplers.py:147:16: DEP004 'torch' imported but declared as a dev dependency
outlines/samplers.py:177:12: DEP004 'torch' imported but declared as a dev dependency
outlines/samplers.py:202:12: DEP004 'torch' imported but declared as a dev dependency
outlines/samplers.py:287:16: DEP004 'torch' imported but declared as a dev dependency
outlines/types/email.py:3:1: DEP003 'typing_extensions' imported but it is a transitive dependency
outlines/types/isbn.py:3:1: DEP003 'typing_extensions' imported but it is a transitive dependency
outlines/types/phone_numbers.py:8:1: DEP003 'typing_extensions' imported but it is a transitive dependency
outlines/types/zip_codes.py:7:1: DEP003 'typing_extensions' imported but it is a transitive dependency
pyproject.toml: DEP002 'datasets' defined as a dependency but not used in the codebase
pyproject.toml: DEP002 'ray' defined as a dependency but not used in the codebase
Found 19 dependency issues.

For more information, see the documentation: https://deptry.com/

This PR proposes to fix these issues. In addition, it proposes to remove pyairports and pycountry, since their imports are wrapped in a try-except block:

try:
    from pyairports.airports import AIRPORT_LIST
except ImportError:
    raise ImportError(
        'The `airports` module requires "pyairports" to be installed. You can install it with "pip install pyairports"'
    )

This PR also proposes to add deptry to the CI/CD pipeline to make sure the list of dependencies stays up to date.

Feel free to let me now your thoughts on this PR, or to close the PR if you think it is not a useful addition.

Disclaimer: I am the author of deptry.

[lapp0]

I like this idea.

We should fix the errors before adding this to the pipeline though. Additionally, would be good to have documentation in docs/community/contribute.md referencing how to run this command. Currently I need to run deptry ./outlines to avoid errors from .myenv/ and build/.

Do these errors mean we would need to explicitly depend on outlines in pyproject.toml?

outlines/__init__.py:2:8: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:3:8: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:4:8: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:5:8: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:6:8: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:7:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:8:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:9:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/__init__.py:10:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/caching.py:51:5: DEP003 'outlines' imported but it is a transitive dependency
outlines/fsm/fsm.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/fsm/guide.py:7:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/fsm/guide.py:8:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/fsm/guide.py:9:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/fsm/parsing.py:38:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/function.py:7:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/api.py:5:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/api.py:6:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/cfg.py:3:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/cfg.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/cfg.py:5:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/choice.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/choice.py:5:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/choice.py:6:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/format.py:3:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/format.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/format.py:5:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/format.py:6:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/fsm.py:3:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/fsm.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/fsm.py:5:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/json.py:7:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/json.py:8:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/json.py:9:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/json.py:10:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:3:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:5:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:6:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:7:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:8:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:9:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:10:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:49:5: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:61:5: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/regex.py:73:5: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/text.py:3:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/text.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/text.py:5:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/generate/text.py:6:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/processors/base_logits_processor.py:11:16: DEP001 'mlx' imported but missing from the dependency definitions
outlines/processors/base_logits_processor.py:96:24: DEP001 'mlx' imported but missing from the dependency definitions
outlines/processors/structured.py:34:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/processors/structured.py:35:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/processors/structured.py:36:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/serve/serve.py:38:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/serve/vllm.py:1:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/types/locales.py:3:1: DEP003 'outlines' imported but it is a transitive dependency
outlines/types/locales.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
Found 61 dependency issues.

[lapp0]

pyairports is necessary for import outlines. It's wrapped in a try-except block, but the block doesn't pass.

[fpgmaas]

I see. then these try-except blocks can be removed, right? Since it will now never enter the except part; in order for people to be able to run that code, they have to install outlines, which has pyairports and pycountry as a direct dependency.

I added both pyairports and pycountry back to the dependencies. Let me know if you want me to make any further changes regarding this, e.g. remove the try-except block.

[fpgmaas]

I like this idea.

We should fix the errors before adding this to the pipeline though. Additionally, would be good to have documentation in docs/community/contribute.md referencing how to run this command. Currently I need to run deptry ./outlines to avoid errors from .myenv/ and build/.

Do these errors mean we would need to explicitly depend on outlines in pyproject.toml?

outlines/__init__.py:2:8: DEP003 'outlines' imported but it is a transitive dependency
[...]
outlines/types/locales.py:4:1: DEP003 'outlines' imported but it is a transitive dependency
Found 61 dependency issues.

The reason for this, is that when deptry is run within the outlines directory, it cannot find the outlines directory itself. The correct way to solve this would be to update the extend-exclude configuration to not look for imports in the .myenv and build directories. I have updated the PR accordingly.

[rlouf]

Thank you for contributing! We are not willing to add additional checks in CI just now, but will re-open should this change.