Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add a composite action for library generation #3173

Merged
merged 60 commits into from
Sep 12, 2024

Conversation

JoeWang1127
Copy link
Collaborator

@JoeWang1127 JoeWang1127 commented Sep 6, 2024

In this PR:

  • Add a composite action to execute library generation shell script.
  • Separate repo specific logic, e.g., installing modules, building images, etc., in the generation shell script.

An example workflow in google-cloud-java: googleapis/google-cloud-java#11117

@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Sep 6, 2024
@@ -0,0 +1,51 @@
name: Copy library generation script
description: Copy library generation shell script to the calling workflow
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: Copy library generation shell script to the calling workflow
description: Copies library generation shell script to the calling workflow

Just a quick nit: In general, descriptions should not be imperative statements but rather assertions of what the file/object/etc does.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the name and description.

.github/scripts/action.yaml Show resolved Hide resolved
.github/scripts/action.yaml Outdated Show resolved Hide resolved
run: |
if [[ "${GITHUB_REPOSITORY}" != "${REPO_FULL_NAME}" ]]; then
echo "This PR comes from a fork. Skip library generation."
exit 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible that this exit 0 just means for the runner that the step finished successfully?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should fail this step?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think any way of stopping the workflow without throwing an error would work fine. I added this comment because I think this exit 0 will be considered as a successful step and the next step would then start. Maybe using GITHUB_OUTPUT along with if in the following steps may do the job (SO)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a separate job to determine whether to run the library generation.

If the pull request comes from a fork, the generation will be skipped.

@JoeWang1127
Copy link
Collaborator Author

The following error occurs if I removed the local install.

Error: ] Some problems were encountered while processing the POMs:
Error:  Non-resolvable import POM: Could not find artifact com.google.cloud:third-party-dependencies:pom:3.35.1-SNAPSHOT in google-maven-central-copy (https://maven-central.storage-download.googleapis.com/maven2) @ line 61, column 19
 @ 
Error:  The build could not read 1 project -> [Help 1]
Error:    
Error:    The project com.google.api.grpc:google-common-protos-parent:2.44.1-SNAPSHOT (/workspace/java-common-protos/pom.xml) has 1 error
Error:      Non-resolvable import POM: Could not find artifact com.google.cloud:third-party-dependencies:pom:3.35.1-SNAPSHOT in google-maven-central-copy (https://maven-central.storage-download.googleapis.com/maven2) @ line 61, column 19 -> [Help 2]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
Error:  [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
Library postprocessing failed
Error: Process completed with exit code 1.

.github/scripts/action.yaml Show resolved Hide resolved
description: the tag of hermetic build image
required: false
token:
description: PAT
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is PAT?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personal access token.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the full name instead of abbreviations since it's not common? See style guide.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to full name.

# run hermetic code generation docker image.
docker run \
--rm \
-u "$(id -u):$(id -g)" \
-v "$(pwd):${workspace_name}" \
-v "$HOME"/.m2:/home/.m2 \
-e GENERATOR_VERSION="${generator_version}" \
-v "${m2_folder}":/home/.m2 \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So .m2 is still needed?

fetch-depth: 0
token: ${{ secrets.CLOUD_JAVA_BOT_TOKEN }}
- name: Generate changed libraries
- name: Determine whether the pull request comes from a fork
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep this step to prevent this workflow runs in a fork.

The workflow will fail if it's a fork and it can prevent us from setting this workflow as required, e.g., renovate PR will be blocked by the failure.

Copy link

sonarcloud bot commented Sep 11, 2024

Copy link

sonarcloud bot commented Sep 11, 2024

Quality Gate Passed Quality Gate passed for 'java_showcase_integration_tests'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

echo "This PR comes from a fork. Skip library generation."
echo "SHOULD_RUN=false" >> $GITHUB_ENV
else
echo "SHOULD_RUN=true" >> $GITHUB_ENV
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to exit the workflow early so we don't have to pass this env to all the subsequent steps?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like another way of skipping steps is creating a separate job and make the generation job depends on that, just like what we've done in unit/integration tests.

I'll keep it as-is now.

@JoeWang1127 JoeWang1127 merged commit 593e2f0 into main Sep 12, 2024
48 of 49 checks passed
@JoeWang1127 JoeWang1127 deleted the chore/add-reusable-workflow branch September 12, 2024 13:25
ldetmer pushed a commit that referenced this pull request Sep 17, 2024
In this PR:
- Add a composite action to execute library generation shell script.
- Separate repo specific logic, e.g., installing modules, building
images, etc., in the generation shell script.

An example workflow in google-cloud-java:
googleapis/google-cloud-java#11117
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants