First stab at implementing the resolver provider using pip internals

[uranusjr]

Migrated from #7789, different branch for collaboration.

TODO:

• Apply the suggested changes from 7789.

[pradyunsg]

Heads up: @pfmoore @uranusjr I think we should break this PR into smaller PRs; before merging into master. :)

[pfmoore]

@pradyunsg I'm not sure why? It's a bunch of new files - how would it be reasonable to split those up?

As it stands, it doesn't even link into the main code in pip. As and when we integrate it, then we may well need to look at doing that integration in an incremental manner (@uranusjr discussed this briefly on Tuesday) but for just putting the new resolvelib objects in place, I think a single PR is fine.

[uranusjr]

Maybe eventually we want to merge the resolver implementation as smaller PRs, likely in the steps like:

1. Merge the refactorings in separate PRs.
2. Merge all the new code (probably OK to be in one bunch).
3. Merge the change that adds the switch to use the new resolver.

But we’ll need to get a working implementation first to know what the necessary refactorings are. This PR is more geared into that.

[pfmoore]

@uranusjr That sounds about right to me.

[pfmoore]

Got the test closer to working again. It still fails with

self = <pip._internal.resolution.resolvelib.candidates.RemoteCandidate object at 0x00000213FE9C9C10>
ireq = <InstallRequirement object: simple from file:///C:/Users/Gustav/AppData/Local/Temp/pytest-of-Gustav/pytest-60/test_resolvelib0/data/packages/simple-1.0.tar.gz editable=False>
dist = simple 1.0 (C:\Users\Gustav\AppData\Local\Temp\pip-temp-dww2jf97\simple\pip-egg-info)

    def __init__(self, ireq, dist):
        # type: (InstallRequirement, Distribution) -> None
        assert ireq.link is not None, "Candidate should be pinned"
        assert ireq.req is not None, "Un-specified requirement not allowed"
>       assert ireq.req.url is not None, "Candidate should be pinned"
E       AssertionError: Candidate should be pinned

.tox\py38\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py:87: AssertionError

which I think is a genuine failure, in the sense that it's a conflict of assumptions. It may be that it's the test not creating the correct "stuff", or it may be that the assertion isn't correct - I can't tell right now. @uranusjr I've committed this as is so you can take a look if you have a moment - it might be obvious to you what's wrong.

[uranusjr]

It seems I misunderstood how InstallRequirement keeps its internal members consistent. I thought it’d set the URL when a link is assigned to it, but apparently that’s not the case.

[uranusjr]

Removing the assertion gives me a new error, probably caused by the cloned InstallRequirement instances using the same build directory…?

with indent_log():
    # Since source_dir is only set for editable requirements.
    assert req.source_dir is None
    req.ensure_has_source_dir(self.build_dir, autodelete_unpacked)
    # If a checkout exists, it's unwise to keep going.  version
    # inconsistencies are logged later, but do not fail the
    # installation.
    # FIXME: this won't upgrade when there's an existing
    # package unpacked in `req.source_dir`
    if os.path.exists(os.path.join(req.source_dir, 'setup.py')):
        raise PreviousBuildDirError(
            "pip can't proceed with requirements '{}' due to a"
            " pre-existing build directory ({}). This is "
            "likely due to a previous installation that failed"
            ". pip is being responsible and not assuming it "
            "can delete this. Please delete it and try again."
            .format(req, req.source_dir)
        )
E       pip._internal.exceptions.PreviousBuildDirError: pip can't proceed with requirements
'simple from file:///C:/Users/uranusjr/AppData/Local/Temp/pytest-of-uranusjr/pytest-1/test_resolvelib0/data/packages/simple-2.0.tar.gz'
due to a pre-existing build directory (C:\Users\uranusjr\AppData\Local\Temp\pip-temp-1qio50zu\simple).
This is likely due to a previous installation that failed. pip is being responsible and not
assuming it can delete this. Please delete it and try again.

.nox\test-3-8\lib\site-packages\pip\_internal\operations\prepare.py:447: PreviousBuildDirError

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

[uranusjr]

I’m going to go ahead and close this PR since is has served its purposes :) Continuing the discussion on the split-up PRs.

[pradyunsg]

I'm deleting the corresponding branch for this PR now.

In the off chance someone wants to look it up, the current ref points to: 5265557