Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow absolute path for the external script #2565

Merged
merged 9 commits into from
Jun 6, 2024
Merged

allow absolute path for the external script #2565

merged 9 commits into from
Jun 6, 2024

Conversation

yhwen
Copy link
Collaborator

@yhwen yhwen commented May 7, 2024 •
edited
Loading

Fixes # .

Description

  • Allow to use the absolute path for the external script.
  • If the external script exists in the job API running machine, the external script will be packaged into the job config folder.
  • If the external script does not exists in the job API running machine, it will be treated as a pre-installed script on the job running machine. The job creation API will skip the packaging of this script. The job execution will load it from the job execution runtime.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Quick tests passed locally by running ./runtest.sh.
  • In-line docstrings updated.
  • Documentation updated.

chesterxgchen
chesterxgchen requested changes May 7, 2024
View reviewed changes
Copy link
Collaborator

@chesterxgchen chesterxgchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add one question, suggest changes

holgerroth
holgerroth requested changes May 7, 2024
View reviewed changes
Copy link
Collaborator

@holgerroth holgerroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look like the right approach to me.

nvflare/job_config/fed_job_config.py Outdated Show resolved Hide resolved
@yhwen
Copy link
Collaborator Author

yhwen commented May 9, 2024

This doesn't look like the right approach to me.

Changed to the following behavior: (Added the unit tests)

  1. When adding an absolute path external script, user also needs to provide the package path to that external script to generate the proper module codes.
  2. If the external script exists in the job API running machine, the external script will be packaged into the job config folder.
  3. If the external script does not exists in the job API running machine, it will be treated as a pre-installed script on the job running machine. The job creation API will skip the packaging of this script. The job execution will load it from the job execution runtime.

holgerroth
holgerroth approved these changes May 30, 2024
View reviewed changes
Copy link
Collaborator

@holgerroth holgerroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Nice work.

@chesterxgchen
Copy link
Collaborator

The PR title ("not allow ..." ) and description ("allow ...") are opposite

@yhwen
Copy link
Collaborator Author

yhwen commented Jun 3, 2024

/build

@yhwen yhwen changed the title not allow absolute path for the external script allow absolute path for the external script Jun 3, 2024
@yhwen
Copy link
Collaborator Author

yhwen commented Jun 3, 2024

The PR title ("not allow ..." ) and description ("allow ...") are opposite

updated.

@yhwen
Copy link
Collaborator Author

yhwen commented Jun 4, 2024

/build

@yhwen yhwen merged commit e37ff67 into NVIDIA:main Jun 6, 2024
16 checks passed
nvidianz pushed a commit to nvidianz/NVFlare that referenced this pull request Jul 8, 2024
@yhwen @YuanTingHsieh @nvidianz
allow absolute path for the external script (NVIDIA#2565)
d1a62a9 
* add check not allow absolute path for the external script in job creation API.

* Changed the job creation API to allow absolution path ext_scripts, but not copy the script.

* Added absolute path ext_script support for job creation.

* removed the no use import.

* Added handle for the line continous support, added unit test.

* Removed the requirement for absolute external script to have the package path.

* enhance the handling of multi-lines import. added more cases of unit test.

---------

Co-authored-by: Yuan-Ting Hsieh (謝沅廷) <yuantingh@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@holgerroth holgerroth holgerroth approved these changes

@chesterxgchen chesterxgchen Awaiting requested review from chesterxgchen

@yanchengnv yanchengnv Awaiting requested review from yanchengnv

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@yhwen @chesterxgchen @holgerroth @yanchengnv @YuanTingHsieh