Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK is broken when installed by git+https #2422

Closed
Electronic-Waste opened this issue Sep 5, 2024 · 4 comments · Fixed by #2424
Closed

SDK is broken when installed by git+https #2422

Electronic-Waste opened this issue Sep 5, 2024 · 4 comments · Fixed by #2424

Comments

@Electronic-Waste
Copy link
Member

Electronic-Waste commented Sep 5, 2024

What happened?

When I run the following code with kubeflow-katib installed by: pip install git+https://github.com/kubeflow/katib.git@master#subdirectory=sdk/python/v1beta1

import kubeflow.katib as katib

# Step 1. Create an objective function with push-based metrics collection.
def objective(parameters):
    # Import required packages.
    import time
    import kubeflow.katib as katib
    time.sleep(5)
    # Calculate objective function.
    result = 4 * int(parameters["a"]) - float(parameters["b"]) ** 2
    # Push metrics to Katib DB.
    katib.report_metrics({"result": result})

# Step 2. Create HyperParameter search space.
parameters = {
    "a": katib.search.int(min=10, max=20),
    "b": katib.search.double(min=0.1, max=0.2)
}

# Step 3. Create Katib Experiment with 4 Trials and 2 CPUs per Trial.
# We choose to install the latest changes of Python SDK because `report_metrics` has not been
# supported yet. Thus, the base image must have `git` command to download the package.
katib_client = katib.KatibClient(namespace="kubeflow")
name = "tune-experiment"
katib_client.tune(
    name=name,
    objective=objective,
    parameters=parameters,
    base_image="electronicwaste/push-metrics-collector:v0.0.9", # python:3.11-slim + git
    objective_metric_name="result",
    max_trial_count=4,
    resources_per_trial={"cpu": "2"},
    packages_to_install=["git+https://github.com/kubeflow/katib.git@master#subdirectory=sdk/python/v1beta1"],
    # packages_to_install=["kubeflow-katib==0.18.0"],
    metrics_collector_config={"kind": "Push"},
)

# Step 4. Wait until Katib Experiment is complete
katib_client.wait_for_experiment_condition(name=name)

# Step 5. Get the best HyperParameters.
print(katib_client.get_optimal_hyperparameters(name))

An error occurred:

Traceback (most recent call last):
  File "/home/ws/katib-example/push.py", line 1, in <module>
    import kubeflow.katib as katib
  File "/home/ws/miniconda3/envs/katib/lib/python3.10/site-packages/kubeflow/katib/__init__.py", line 73, in <module>
    from kubeflow.katib.api.katib_client import KatibClient
  File "/home/ws/miniconda3/envs/katib/lib/python3.10/site-packages/kubeflow/katib/api/katib_client.py", line 27, in <module>
    from kubeflow.katib.types.trainer_resources import TrainerResources
ModuleNotFoundError: No module named 'kubeflow.katib.types'

And I went to dir /home/ws/miniconda3/envs/katib/lib/python3.10/site-packages/kubeflow/katib/, finding that dir types was missing:

(katib)  ws@master1  ~/miniconda3/envs/katib/lib/python3.10/site-packages/kubeflow/katib  ls
api            configuration.py  exceptions.py  katib_api_pb2_grpc.py  models       rest.py
api_client.py  constants         __init__.py    katib_api_pb2.py       __pycache__  utils

What did you expect to happen?

The code should be executed without error when I installed the SDK with:

pip install git+https://github.com/kubeflow/katib.git@a524f33830e02189476efaf6d9045cbd2ce605f0#subdirectory=sdk/python/v1beta1

Environment

Kubernetes version:

$ kubectl version
Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.1

Katib controller version:

$ kubectl get pods -n kubeflow -l katib.kubeflow.org/component=controller -o jsonpath="{.items[*].spec.containers[*].image}"
docker.io/kubeflowkatib/katib-controller:latest

Katib Python SDK version:

$ pip show kubeflow-katib
Name: kubeflow-katib
Version: 0.17.0
Summary: Katib Python SDK for APIVersion v1beta1
Home-page: https://github.com/kubeflow/katib/tree/master/sdk/python/v1beta1
Author: Kubeflow Authors
Author-email: premnath.vel@gmail.com
License: Apache License Version 2.0
Location: /home/ws/miniconda3/envs/katib/lib/python3.10/site-packages
Requires: certifi, grpcio, kubernetes, protobuf, setuptools, six, urllib3
Required-by:

Impacted by this bug?

Give it a 👍 We prioritize the issues with most 👍

@Electronic-Waste
Copy link
Member Author

/area sdk
/good-first-issue

Copy link

@Electronic-Waste:
This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

In response to this:

/area sdk
/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@google-oss-prow google-oss-prow bot added area/sdk good first issue Good for newcomers help wanted Extra attention is needed labels Sep 5, 2024
@andreyvelich
Copy link
Member

@helenxie-bit Can you fix this bug please ?
I think, we should add __init__.py file in the types folder: https://github.com/kubeflow/katib/tree/master/sdk/python/v1beta1/kubeflow/katib/types

@helenxie-bit
Copy link
Contributor

@helenxie-bit Can you fix this bug please ? I think, we should add __init__.py file in the types folder: https://github.com/kubeflow/katib/tree/master/sdk/python/v1beta1/kubeflow/katib/types

@andreyvelich @Electronic-Waste I'm sorry about that. I will fix it as soon as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants