Enable installation of vsphere-automation-sdk-python with poetry

[arthurzenika]

Describe the bug

1. poetry add vsphere-automation-sdk gives :

Using version ^0.0.1 for vsphere-automation-sdk

Updating dependencies
Resolving dependencies... (5.0s)

Writing lock file

Package operations: 0 installs, 1 update, 0 removals

  • Updating vsphere-automation-sdk (1.80.0 /home/arthur/.cache/pypoetry/artifacts/8b/50/54/34584a8a6f6286a2d0bd0041a55cb559c54ba29adb9ccb97d25a384d94/v8.0.0.1.tar.gz -> 0.0.1): Failed

  CalledProcessError

  Command '['/home/arthur/src/example-project/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/home/arthur/src/example-project/.venv', '--upgrade', '--no-deps', '/home/arthur/.cache/pypoetry/artifacts/42/b1/df/65279f2bde3d295c08b7dce183d8aabf7a8f74f14c6bf655d95035c5dd/vsphere-automation-sdk-0.0.1.tar.gz']' returned non-zero exit status 1.

  at /usr/lib/python3.10/subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│ 
       528│ 

The following error occurred when trying to handle this error:


  EnvCommandError

  Command ['/home/arthur/src/example-project/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/home/arthur/src/example-project/.venv', '--upgrade', '--no-deps', '/home/arthur/.cache/pypoetry/artifacts/42/b1/df/65279f2bde3d295c08b7dce183d8aabf7a8f74f14c6bf655d95035c5dd/vsphere-automation-sdk-0.0.1.tar.gz'] errored with the following return code 1, and output: 
  Processing /home/arthur/.cache/pypoetry/artifacts/42/b1/df/65279f2bde3d295c08b7dce183d8aabf7a8f74f14c6bf655d95035c5dd/vsphere-automation-sdk-0.0.1.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'error'
    error: subprocess-exited-with-error
    
    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> [18 lines of output]
        Traceback (most recent call last):
          File "/home/arthur/src/example-project/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
            main()
          File "/home/arthur/src/example-project/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/home/arthur/src/example-project/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
            return hook(config_settings)
          File "/tmp/pip-build-env-sh5yoj9h/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=['wheel'])
          File "/tmp/pip-build-env-sh5yoj9h/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
            self.run_setup()
          File "/tmp/pip-build-env-sh5yoj9h/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 484, in run_setup
            super(_BuildMetaLegacyBackend,
          File "/tmp/pip-build-env-sh5yoj9h/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
            exec(code, locals())
          File "<string>", line 13, in <module>
          File "<string>", line 5, in read
        FileNotFoundError: [Errno 2] No such file or directory: 'LICENSE.txt'
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  

  at ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry/utils/env.py:1540 in _run
      1536│                 output = subprocess.check_output(
      1537│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1538│                 )
      1539│         except CalledProcessError as e:
    → 1540│             raise EnvCommandError(e, input=input_)
      1541│ 
      1542│         return decode(output)
      1543│ 
      1544│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install /home/arthur/.cache/pypoetry/artifacts/42/b1/df/65279f2bde3d295c08b7dce183d8aabf7a8f74f14c6bf655d95035c5dd/vsphere-automation-sdk-0.0.1.tar.gz

  at ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry/utils/pip.py:58 in pip_install
       54│ 
       55│     try:
       56│         return environment.run_pip(*args)
       57│     except EnvCommandError as e:
    →  58│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       59│ 

Reproduction steps

1. install poetry https://python-poetry.org/
2. poetry add vsphere-automation-sdk

Expected behavior

poetry add vsphere-automation-sdk to work, and versions be coherent between github and pypi.

Additional context

Related : #354 and #200

[arthurzenika]

With poetry 1.1 we get a different error :

python -m poetry install
Updating dependencies
Resolving dependencies... (16.4s)

  SolverProblemError

  Because vsphere-automation-sdk (1.80.0) depends on vmc-draas-client-bindings (1.20.0) which doesn't match any versions, vsphere-automation-sdk is forbidden.
  So, because cmdb-pic-tooling depends on vsphere-automation-sdk (1.80.0), version solving failed.

  at /usr/lib/python3/dist-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│ 
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

Again https://pypi.org/project/vmc-draas-client-bindings/#history only shows a 0.0.1 version...

[aagrawal3]

Duplicate of #326

[arthurzenika]

Here is a workaround for anyone else using poetry. In your pyproject.toml :

[tool.poetry.dependencies]
# vsphere-automation-sdk work around
vsphere-automation-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/archive/refs/tags/v8.0.0.1.tar.gz"}
vmc-draas-client-bindings = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/vmc-draas-client-bindings/vmc_draas_client_bindings-1.20.0-py2.py3-none-any.whl"}
vapi-runtime = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/vapi-runtime/vapi_runtime-2.37.0-py2.py3-none-any.whl"}
vapi-common-client = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/vapi-common-client/vapi_common_client-2.37.0-py2.py3-none-any.whl"}
nsx-policy-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/nsx-policy-python-sdk/nsx_policy_python_sdk-4.0.1.0.0-py2.py3-none-any.whl"}
nsx-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/nsx-python-sdk/nsx_python_sdk-4.0.1.0.0-py2.py3-none-any.whl"}
nsx-vmc-aws-integration-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/nsx-vmc-aws-integration-python-sdk/nsx_vmc_aws_integration_python_sdk-4.0.1.0.0-py2.py3-none-any.whl"}
nsx-vmc-policy-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/nsx-vmc-policy-python-sdk/nsx_vmc_policy_python_sdk-4.0.1.0.0-py2.py3-none-any.whl"}
vapi-client-bindings = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/vapi-client-bindings/vapi_client_bindings-4.0.0-py2.py3-none-any.whl"}
vmc-client-bindings = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/master/lib/vmc-client-bindings/vmc_client_bindings-1.61.0-py2.py3-none-any.whl"}
# end of vsphere-automation-sdk work around

[AnatomicJC]

Thank you @arthurzenika for your workaround. It is now possible for me to setup this framework with poetry 💪

Here is the updated workaround list with latest release 8.0.2.0:

# vsphere-automation-sdk work around
vsphere-automation-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/archive/refs/tags/v8.0.2.0.tar.gz"}
nsx-policy-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-policy-python-sdk/nsx_policy_python_sdk-4.1.2.0.0-py2.py3-none-any.whl"}
nsx-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-python-sdk/nsx_python_sdk-4.1.2.0.0-py2.py3-none-any.whl"}
nsx-vmc-aws-integration-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-vmc-aws-integration-python-sdk/nsx_vmc_aws_integration_python_sdk-4.1.2.0.0-py2.py3-none-any.whl"}
nsx-vmc-policy-python-sdk = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-vmc-policy-python-sdk/nsx_vmc_policy_python_sdk-4.1.2.0.0-py2.py3-none-any.whl"}
vapi-common-client = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vapi-common-client/vapi_common_client-2.44.0-py2.py3-none-any.whl"}
vapi-runtime = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vapi-runtime/vapi_runtime-2.44.0-py2.py3-none-any.whl"}
vcenter-bindings = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vcenter-bindings/vcenter_bindings-4.2.0-py2.py3-none-any.whl"}
vmwarecloud-aws = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vmwarecloud-aws/vmwarecloud_aws-1.64.0-py2.py3-none-any.whl"}
vmwarecloud-draas = {url = "https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vmwarecloud-draas/vmwarecloud_draas-1.23.0-py2.py3-none-any.whl"}
# end of vsphere-automation-sdk work around

[AnatomicJC]

I asked to ChatGPT to write a python script to automate the dependency list, it takes vsphere-automation-sdk-python tag as first arg to generate the list.

$ python script.py v8.0.2.0
# vsphere-automation-sdk work around
vsphere-automation-sdk = {url = https://github.com/vmware/vsphere-automation-sdk-python/archive/refs/tags/v8.0.2.0.tar.gz}
nsx-policy-python-sdk = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-policy-python-sdk/nsx_policy_python_sdk-4.1.2.0.0-py2.py3-none-any.whl}
nsx-python-sdk = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-python-sdk/nsx_python_sdk-4.1.2.0.0-py2.py3-none-any.whl}
nsx-vmc-aws-integration-python-sdk = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-vmc-aws-integration-python-sdk/nsx_vmc_aws_integration_python_sdk-4.1.2.0.0-py2.py3-none-any.whl}
nsx-vmc-policy-python-sdk = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/nsx-vmc-policy-python-sdk/nsx_vmc_policy_python_sdk-4.1.2.0.0-py2.py3-none-any.whl}
vapi-common-client = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vapi-common-client/vapi_common_client-2.44.0-py2.py3-none-any.whl}
vapi-runtime = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vapi-runtime/vapi_runtime-2.44.0-py2.py3-none-any.whl}
vcenter-bindings = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vcenter-bindings/vcenter_bindings-4.2.0-py2.py3-none-any.whl}
vmwarecloud-aws = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vmwarecloud-aws/vmwarecloud_aws-1.64.0-py2.py3-none-any.whl}
vmwarecloud-draas = {url = https://github.com/vmware/vsphere-automation-sdk-python/raw/v8.0.2.0/lib/vmwarecloud-draas/vmwarecloud_draas-1.23.0-py2.py3-none-any.whl}
# end of vsphere-automation-sdk work around

Here is the python script:

import os
import glob
import requests
import argparse

# Parsing the command line argument for the Git tag
parser = argparse.ArgumentParser(description='Generate a list of .whl files from the vsphere-automation-sdk-python repo for a given Git tag.')
parser.add_argument('tag', help='The Git tag to search for .whl files')
args = parser.parse_args()

# Setting up the base URL and the target directory
base_url = "https://github.com/vmware/vsphere-automation-sdk-python"
target_dir = "lib"

def get_whl_files(tag):
    # Construct the URL to download the archive of the given tag
    archive_url = f"{base_url}/archive/refs/tags/{tag}.tar.gz"
    
    # List to hold the paths of .whl files
    whl_files = []

    # Construct the API URL to fetch the contents of the lib directory at the given tag
    api_url = f"https://api.github.com/repos/vmware/vsphere-automation-sdk-python/git/trees/{tag}?recursive=1"

    # Make a request to the GitHub API
    response = requests.get(api_url)
    if response.status_code == 200:
        data = response.json()
        for file in data.get('tree', []):
            # Check if the file is in the lib directory and is a .whl file
            if file['path'].startswith(target_dir) and file['path'].endswith('.whl'):
                # Construct the raw URL for the file
                raw_file_url = f"{base_url}/raw/{tag}/{file['path']}"
                whl_files.append(raw_file_url)
    else:
        print(f"Failed to retrieve data: {response.status_code}")

    return whl_files

def main():
    # Retrieve the .whl files for the given tag
    whl_files = get_whl_files(args.tag)
    
    # Generate the output file content
    output = []
    output.append("# vsphere-automation-sdk work around")
    output.append(f"vsphere-automation-sdk = {{url = \"{base_url}/archive/refs/tags/{args.tag}.tar.gz\"}}")
    for whl in whl_files:
        lib_name = whl.split('/')[-2]
        output.append(f"{lib_name} = {{url = \"{whl}\"}}")
    output.append("# end of vsphere-automation-sdk work around")
    
    # Print the output to the console
    print("\n".join(output))

if __name__ == "__main__":
    main()