Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cross-crossgen job, reduce workitem size #33436

Merged
merged 4 commits into from
Mar 11, 2020

Conversation

jashook
Copy link
Contributor

@jashook jashook commented Mar 10, 2020

The work item should be ~1/2 the size of the older workitem. The largest difference will be the bin/* folder will now be ommitted.

Workitem size goes from 750m to 350m, decided to print the elapsed runtime as well into the console log. Which will help with #33175

@jashook jashook linked an issue Mar 10, 2020 that may be closed by this pull request
@jashook jashook requested review from trylek, echesakov and jkoritzinsky and removed request for trylek March 11, 2020 15:28
@jashook jashook closed this Mar 11, 2020
@jashook jashook reopened this Mar 11, 2020
@jashook
Copy link
Contributor Author

jashook commented Mar 11, 2020

Console looks good to me, run time on device is 7 minutes. Total time is ~14 minutes, so ~7 minute copy and unpack time.

@jashook
Copy link
Contributor Author

jashook commented Mar 11, 2020

@jashook
Copy link
Contributor Author

jashook commented Mar 11, 2020

The sooner this is merged the more capacity we can give back to the lab @echesakovMSFT ptal

Copy link
Contributor

@echesakov echesakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thank you @jashook!

@jashook
Copy link
Contributor Author

jashook commented Mar 11, 2020

The job can put its own results (for native architecture scenario) inside the work item directory. I am talking about results for cross-architecture scenario that are only read during compare step and can be placed inside CorrelationPayloadDirectory .

This makes sense and saves a minute+ Will do this is a follow up change though.

@jashook jashook closed this Mar 11, 2020
@jashook jashook reopened this Mar 11, 2020
@echesakov
Copy link
Contributor

Console looks good to me, run time on device is 7 minutes. Total time is ~14 minutes, so ~7 minute copy and unpack time.

@jashook That probably means that there is no much room left for improvement other than parallelizing across multiple Helix machines?

@jashook jashook merged commit 871e874 into dotnet:master Mar 11, 2020
@jashook
Copy link
Contributor Author

jashook commented Mar 11, 2020

@jashook That probably means that there is no much room left for improvement other than parallelizing across multiple Helix machines?

I still would like to get #33175 merged. I believe we can drop the execution time down by half. In general to me executing on multiple machines seems like a waste if the time to download is larger than the execution time.

@jashook
Copy link
Contributor Author

jashook commented Mar 11, 2020

There is more pruning that can happen in the core_root directory. #33438 tracks this.

@jashook jashook deleted the reduce_crossgen_compairison_size branch March 11, 2020 18:15
@ghost ghost locked as resolved and limited conversation to collaborators Dec 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make work item payload smaller for crossgen-comparison job
4 participants