Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Generate Jenkinsfile from a template #10740

Merged
merged 2 commits into from
Mar 25, 2022

Conversation

driazati
Copy link
Member

@driazati driazati commented Mar 23, 2022

This uses jinja2 to generate the Jenkinsfile. This is useful since it lets us both keep common functionality easy to define (i.e. iterate over all images and do something) while keeping the output easy to debug (you can look at the Jenkinsfile directly instead of trying to imagine what the Groovy interpreter will do). This will become more useful as we start to make CI more configurable, such as adding dynamic test sharding.

This mostly introduces the infrastructure and makes some token changes to demonstrate the generation process, but already its use is shown since the parameters was missing an entry for the ci_hexagon image. This PR intentionally leaves more usage of the templates/variables to follow up PRs in order to keep this PR simple.

cc @areusch

driazati added a commit to driazati/tvm that referenced this pull request Mar 23, 2022
areusch pushed a commit that referenced this pull request Mar 24, 2022
This is needed for #10740

Co-authored-by: driazati <driazati@users.noreply.github.com>
Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redirected to here from #9823. Overall LGTM but currently I could only see one value (i.e., images) in the template. It would be good to list all templatable values (like the goal of this PR or this and follow-up PRs).

jenkins/generate.py Outdated Show resolved Hide resolved
jenkins/generate.py Outdated Show resolved Hide resolved
This uses `jinja2` to generate the Jenkinsfile. This is useful since it lets us both keep common functionality easy to define (i.e. iterate over all images and do something) while keeping the output easy to debug (you can look at the `Jenkinsfile` directly instead of trying to imagine what the Groovy interpreter will do). This will become more useful as we start to make CI more configurable, such as adding dynamic test sharding.

This mostly introduces the infrastructure and makes some token changes to demonstrate the generation process, but already its use is shown since the parameters was missing an entry for the `ci_hexagon` image.
@driazati driazati force-pushed the generate_jenkins branch 2 times, most recently from 56c0530 to 26ccc62 Compare March 25, 2022 00:38
@driazati
Copy link
Member Author

The templated values come from code (https://github.com/apache/tvm/pull/10740/files#diff-670a379669fd3f1cde13bc32dbb2e1ea020fa2ceac1f1c43f4ecf4b6589bfa14R30) the Jenkinsfile.j2 is mostly a copy-paste from the Jenkinsfile, see a proper diff here: https://gist.github.com/2cfb7e11b628149042e9f9ed824ac8e4

This PR doesn't make too many actual changes so we can land it quickly to avoid having to rebase too much

@driazati driazati marked this pull request as ready for review March 25, 2022 00:43
@github-actions github-actions bot requested a review from areusch March 25, 2022 00:44
driazati added a commit to driazati/tvm that referenced this pull request Mar 25, 2022
Copy link
Contributor

@areusch areusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, let's work to merge the new images quickly

@areusch areusch merged commit 1b654e9 into apache:main Mar 25, 2022
driazati added a commit to driazati/tvm that referenced this pull request Mar 25, 2022
pfk-beta pushed a commit to pfk-beta/tvm that referenced this pull request Apr 11, 2022
This is needed for apache#10740

Co-authored-by: driazati <driazati@users.noreply.github.com>
pfk-beta pushed a commit to pfk-beta/tvm that referenced this pull request Apr 11, 2022
* [ci] Generate Jenkinsfile from a template

This uses `jinja2` to generate the Jenkinsfile. This is useful since it lets us both keep common functionality easy to define (i.e. iterate over all images and do something) while keeping the output easy to debug (you can look at the `Jenkinsfile` directly instead of trying to imagine what the Groovy interpreter will do). This will become more useful as we start to make CI more configurable, such as adding dynamic test sharding.

This mostly introduces the infrastructure and makes some token changes to demonstrate the generation process, but already its use is shown since the parameters was missing an entry for the `ci_hexagon` image.

* Address comments, fix CI with temporary workaround

Co-authored-by: driazati <driazati@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants