feat(runs_on): permit using array of string to select specific runner #1740

NikitaCOEUR · 2024-07-08T07:29:22Z

Check List

[X ] Read CONTRIBUTING.md
- [X ] Read OSS Contribution Guide
[X ] Write a GitHub Issue before creating a Pull Request
- Link to the issue: Use array of tags to select specific self hosted github runners as "runs-on" #1738
[X ] All commits are signed
- This repository enables Require signed commits, so all commits must be signed
[X ] Avoid force push
[ X] Do only one thing in one Pull Request

Permit using array of string to select GitHub runners on which run jobs.
Necessary to select self hosted runners.

suzuki-shunsuke · 2024-07-08T12:12:57Z

Your commits aren't singned.

[X ] All commits are signed
This repository enables Require signed commits, so all commits must be signed

The base branch requires all commits to be signed. Learn more about signing commits.

… runner

NikitaCOEUR · 2024-07-08T13:21:07Z

Hi,

I saw that we could sign with our SSH key. I thought that since I was using my SSH key to clone the repo, the associated commits were automatically signed. But indeed, a bit of configuration was missing...

I have redone my commits as indicated in the procedure and force-pushed these two rebased commits with a signature.

I would like to ensure that all the changes I have made work correctly before merging them. It is quite complicated to test due to all the internal references to suzuki-shunsuke/tfaction@main. I saw that we could use a GitHub Action to create the PR and point to it in our workflows?

github-actions · 2024-07-08T13:36:20Z

🚀 Pull Request Branch was created or updated

Build link

The pull request branch pr/1740 was created or updated.

You can try this pull request in your workflows by changing tfaction version to pr/1740.

e.g.

- uses: suzuki-shunsuke/tfaction/setup@pr/1740

To update the pull request branch again, please run the workflow.

suzuki-shunsuke · 2024-07-10T21:29:31Z

@NikitaCOEUR You can test this pull request using the version pr/1740.

feat(runs_on): permit using array of string to select specific runner #1740 (comment)

NikitaCOEUR · 2024-07-10T21:30:52Z

@NikitaCOEUR You can test this pull request using the version pr/1740.

feat(runs_on): permit using array of string to select specific runner #1740 (comment)

Yes thanks I saw that but I couldn't find any time for that now. Maybe next week!

NikitaCOEUR · 2024-07-15T21:02:24Z

Hi,

I've added a new commit to fix an issue I introduced where the runs_on field was no longer optional in certain configurations. This should restore the previous behavior.

Additionally, I've retested everything, but I still encounter an error that I don't understand:

 bash: /home/runner/work/_actions/suzuki-shunsuke/github-action-terraform-init/71673ab29266f67c4470773675e087e9a4dda3c5/main.sh: No such file or directory

Despite this, the changes I've made seem to have the desired effect:

content of my tfaction-root.yaml :

target_groups:

# Target group 1 with specific self hosted runner (specified via array of strings)
- working_directory: terraform/env-TF-ACTION-TEST/
  target: TF-ACTION-TEST/
  environment: TF-ACTION-TEST
  runs_on:
    - self-hosted
    - runners
  secrets: # GitHub Secrets
  - env_name: AWS_ACCESS_KEY_ID # Environment variable name
    secret_name: BACKEND_ACCESS_KEY_ID
  - env_name: AWS_SECRET_ACCESS_KEY # Environment variable name
    secret_name: BACKEND_SECRET_KEY
  - env_name: SOPS_AGE_KEY
    secret_name: SOPS_AGE_KEY

# Target group 2 with specified runners (with string)
- working_directory: terraform/env-TF-ACTION-TEST2/
  target: TF-ACTION-TEST2/
  environment: TF-ACTION-TEST
  runs_on: ubuntu-latest
  secrets: # GitHub Secrets
  - env_name: AWS_ACCESS_KEY_ID # Environment variable name
    secret_name: BACKEND_ACCESS_KEY_ID
  - env_name: AWS_SECRET_ACCESS_KEY # Environment variable name
    secret_name: BACKEND_SECRET_KEY
  - env_name: SOPS_AGE_KEY
    secret_name: SOPS_AGE_KEY

# Target group 3 without any runs_on specified
- working_directory: terraform/env-TF-ACTION-TEST3/
  target: TF-ACTION-TEST3/
  environment: TF-ACTION-TEST
  secrets: # GitHub Secrets
  - env_name: AWS_ACCESS_KEY_ID # Environment variable name
    secret_name: BACKEND_ACCESS_KEY_ID
  - env_name: AWS_SECRET_ACCESS_KEY # Environment  variable name
    secret_name: BACKEND_SECRET_KEY
  - env_name: SOPS_AGE_KEY
    secret_name: SOPS_AGE_KEY

Target group 1 with specific self hosted runner (specified via array of strings)

Target group 2 with specified runners (with string)

Target group 3 without any runs_on specified

Could you please help me with this issue?

Thanks!

github-actions · 2024-07-15T21:48:29Z

🚀 Pull Request Branch was created or updated

Build link

The pull request branch pr/1740 was created or updated.

You can try this pull request in your workflows by changing tfaction version to pr/1740.

e.g.

- uses: suzuki-shunsuke/tfaction/setup@pr/1740

To update the pull request branch again, please run the workflow.

suzuki-shunsuke · 2024-07-15T21:51:58Z

 bash: /home/runner/work/_actions/suzuki-shunsuke/github-action-terraform-init/71673ab29266f67c4470773675e087e9a4dda3c5/main.sh: No such file or directory

I'm not sure why this error occurred. main.sh should exist.

    - run: bash ${{ github.action_path }}/main.sh

https://github.com/suzuki-shunsuke/github-action-terraform-init/blob/71673ab29266f67c4470773675e087e9a4dda3c5/action.yaml#L28

This error may occur if you run this action in a container, but I guess you didn't do that.

github.action_path contains wrong path when used inside a container actions/runner#2185

NikitaCOEUR · 2024-07-15T22:05:50Z

Hmm ..
Indeed, I am using containers to run my GitHub actions.
That seems to be the problem.

I just saw a workaround suggested by someone: actions/runner#716 (comment)

I'm not sure if that would make sense here?

suzuki-shunsuke · 2024-07-15T22:49:56Z

Could you allow me to push commits to the feature branch?

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork

e.g. aquaproj/aqua-registry#24089

Maintainers are allowed to edit this pull request.

NikitaCOEUR · 2024-07-16T05:21:09Z

So, I had to transfer the ownership of the fork to my user account to be able to activate this option.

suzuki-shunsuke · 2024-07-16T05:29:23Z

So, I had to transfer the ownership of the fork to my user account to be able to activate this option.

Oh, really. I didn't know that.
The document seems not to refer that.

Anyway, thank you for your update.

suzuki-shunsuke · 2024-07-16T05:38:43Z

a0f260e Replaced ${{github.action_path}} with $GITHUB_ACTION_PATH.

github-actions · 2024-07-16T05:42:43Z

🚀 Pull Request Branch was created or updated

Build link

The pull request branch pr/1740 was created or updated.

You can try this pull request in your workflows by changing tfaction version to pr/1740.

e.g.

- uses: suzuki-shunsuke/tfaction/setup@pr/1740

To update the pull request branch again, please run the workflow.

NikitaCOEUR · 2024-07-16T22:06:41Z

The purpose of this pull request seems fine.

However, I have issues with the Taskfile combined with the execution of my GitHub Actions in a container.

I use a Taskfile at the root of my project as a wrapper for Terraform commands. This is used when a task command is executed from the root or in one of the subfolders. But they have implemented a safeguard based on directory ownership; task ascends through parent directories and stops if it does not find a Taskfile or as soon as the owner of the current directory changes compared to the previous one(https://github.com/go-task/task/blob/573949573968cc7056b98e0fdbee1d145ad869a9/taskfile/taskfile.go#L149) ... And this is the case since the containers on GitHub Actions are executed with the root user...

Also, the repo is cloned on the runner, and the volume is mounted in the container. When accessing the repo's folders, the root folder of the repo belongs to the user running Docker, but the subfolders belong to the user of the container (root).

NikitaCOEUR · 2024-07-16T22:17:34Z

It seems that a workaround exists : actions/checkout#47 (comment)

I'll test that tomorrow.

suzuki-shunsuke · 2024-07-16T22:22:05Z

The purpose of this pull request seems fine.

Thank you for testing. It looks good.
Seems like your issue has nothing to do with tfaction.

NikitaCOEUR · 2024-07-17T21:20:44Z

It seems that a workaround exists : actions/checkout#47 (comment)

I'll test that tomorrow.
Doesn't work )=

But I have a workaround for my problem with Taskfile, which I posted in an issue: go-task/task#1683 (comment)

However, this problem appears again with the git command. I just opened an issue here: suzuki-shunsuke/github-action-terraform-init#19

suzuki-shunsuke · 2024-07-18T09:45:29Z

However, this problem appears again with the git command. I just opened an issue here: suzuki-shunsuke/github-action-terraform-init#19

The issue was solved.

[BUG] This action inside a workflow executed inside a container doesn't work github-action-terraform-init#19 (comment)

So everything is fine, right?

NikitaCOEUR · 2024-07-18T22:12:53Z

Do you have some doc to update for this feature ?

suzuki-shunsuke · 2024-07-18T22:51:27Z

Do you have some doc to update for this feature ?

~/repos/src/github.com/suzuki-shunsuke/tfaction-docs main                                                                             07:49:57
$ git grep runs_on
docs/config/tfaction-root-yaml.md:  runs_on: ubuntu-latest # default is "ubuntu-latest". This is useful to use GitHub Actions Self Hosted Runner for the specific provider

https://github.com/suzuki-shunsuke/tfaction-docs/blob/bfc28814d4acfaf8d535de76509affe052a34d8e/docs/config/tfaction-root-yaml.md?plain=1#L177

suzuki-shunsuke · 2024-07-18T22:54:37Z

✅ I'll update the document.

fix the example of runs_on tfaction-docs#406

NikitaCOEUR · 2024-07-18T22:58:01Z

I can't ensure non-regression without running within a container. I have an issue; I don't know if it's caused by the transition from ${{GitHub.action_path}} to $GITHUB_ACTION or if it's on my end.

suzuki-shunsuke · 2024-07-18T22:59:42Z

I have an issue

Could you explain the detail of your issue?

suzuki-shunsuke · 2024-07-18T23:01:27Z

I can't ensure non-regression without running within a container.

I've already confirmed this pr worked fine without a container.

NikitaCOEUR · 2024-07-18T23:06:54Z

I have an issue

Could you explain the detail of your issue?

No, it's on my end then; the problem only occurs when I run without a container. I get an exit code 2 when running Terraform through my wrapper. But it must be due to the context change. I use the action within a container anyway, so there's no problem for me.

suzuki-shunsuke · 2024-07-18T23:10:59Z

schema/tfaction-root.json

        }
      }
    },
    "Envs": {
      "type": "object",
      "description": "environment variables"
+    },
+    "RunsOn": {
+      "type": ["string", "array"],


I'm not familiar with JSON Schema, so I'm not sure if this is correct.
I'll look into.

https://cswr.github.io/JsonSchema/spec/multiple_types/
It seems correct, no ?

📝 type can accept an array.

https://json-schema.org/draft/2020-12/json-schema-core

"type": ["object", "boolean"],

But I think array is ambiguous because the type of elements is unknown. 🤔

Maybe this is better :

"RunsOn": { "oneOf": [ { "type": "string" }, { "type": "array", "items": { "type": "string" } }, { "type": "null" } ], "description": "The type of runner that the job will run on" }

https://json-schema.org/understanding-json-schema/reference/combining#anyOf

Should we use anyOf?

"RunsOn": { "anyOf": [ { "type": "string" }, { "type": "array", "items": { "type": "string" } } ], "description": "The type of runner that the job will run on" }

After reading more, Anyof seems more appropriate. It's more like a OR and not a XOR. So yes definitely.

But null type is necessary because we can let empty the property no?

If runs_on is not set, the type definition is unrelated. So I don't think we need to add null.
If we want to support the setting which sets null explictly, we need to add null.
But I don't think we need to support null explicitly.

runs_on: null

Fixed. 7290811

suzuki-shunsuke · 2024-07-19T03:23:58Z

v1.5.2 is out 🎉
https://github.com/suzuki-shunsuke/tfaction/releases/tag/v1.5.2

NikitaCOEUR mentioned this pull request Jul 8, 2024

feat(runs_on): permit array of tags to select runs-on #1739

Closed

NikitaCOEUR changed the title ~~feat(runs_on): permit using array of string to select specific githug…~~ feat(runs_on): permit using array of string to select specific runner Jul 8, 2024

NikitaCOEUR added 2 commits July 8, 2024 15:07

feat(runs_on): permit using array of string to select specific githug…

891baef

… runner

docs(schema): add runs_on to schema

e21e74f

NikitaCOEUR force-pushed the feat/array_for_runs-on branch from 19fa51e to e21e74f Compare July 8, 2024 13:08

fix(runs_on): ensure runs_on can be optional as before

560de74

This was referenced Jul 15, 2024

fix: replace github.action_path with GITHUB_ACTION_PATH suzuki-shunsuke/github-action-terraform-init#18

Merged

Replace github.action_path with GITHUB_ACTION_PATH suzuki-shunsuke/tasks#8

Closed

fix: replace github.action_path with GITHUB_ACTION_PATH

a0f260e

chore: update terraform-fmt and terraform-init actions

deb247e

suzuki-shunsuke reviewed Jul 18, 2024

View reviewed changes

fix: update JSON Schema of tfaction-root.yaml to use anyOf for runs_on

7290811

suzuki-shunsuke merged commit 41b5167 into suzuki-shunsuke:main Jul 19, 2024
12 checks passed

suzuki-shunsuke added the bug Something isn't working label Jul 19, 2024

suzuki-shunsuke added this to the v1.5.2 milestone Jul 19, 2024

suzuki-shunsuke linked an issue Jul 19, 2024 that may be closed by this pull request

Use array of tags to select specific self hosted github runners as "runs-on" #1738

Closed

feat(runs_on): permit using array of string to select specific runner #1740

feat(runs_on): permit using array of string to select specific runner #1740

Conversation

NikitaCOEUR commented Jul 8, 2024

Check List

suzuki-shunsuke commented Jul 8, 2024 • edited Loading

NikitaCOEUR commented Jul 8, 2024

github-actions bot commented Jul 8, 2024

🚀 Pull Request Branch was created or updated

suzuki-shunsuke commented Jul 10, 2024

NikitaCOEUR commented Jul 10, 2024

NikitaCOEUR commented Jul 15, 2024

Target group 1 with specific self hosted runner (specified via array of strings)

Target group 2 with specified runners (with string)

Target group 3 without any runs_on specified

github-actions bot commented Jul 15, 2024

🚀 Pull Request Branch was created or updated

suzuki-shunsuke commented Jul 15, 2024 • edited Loading

NikitaCOEUR commented Jul 15, 2024

suzuki-shunsuke commented Jul 15, 2024 • edited Loading

NikitaCOEUR commented Jul 16, 2024

suzuki-shunsuke commented Jul 16, 2024

suzuki-shunsuke commented Jul 16, 2024 • edited Loading

github-actions bot commented Jul 16, 2024

🚀 Pull Request Branch was created or updated

NikitaCOEUR commented Jul 16, 2024 • edited Loading

NikitaCOEUR commented Jul 16, 2024

suzuki-shunsuke commented Jul 16, 2024

NikitaCOEUR commented Jul 17, 2024

suzuki-shunsuke commented Jul 18, 2024

NikitaCOEUR commented Jul 18, 2024

suzuki-shunsuke commented Jul 18, 2024

suzuki-shunsuke commented Jul 18, 2024 • edited Loading

NikitaCOEUR commented Jul 18, 2024

suzuki-shunsuke commented Jul 18, 2024

suzuki-shunsuke commented Jul 18, 2024

NikitaCOEUR commented Jul 18, 2024

suzuki-shunsuke Jul 18, 2024

Choose a reason for hiding this comment

NikitaCOEUR Jul 18, 2024

Choose a reason for hiding this comment

suzuki-shunsuke Jul 18, 2024

Choose a reason for hiding this comment

NikitaCOEUR Jul 18, 2024

Choose a reason for hiding this comment

suzuki-shunsuke Jul 18, 2024

Choose a reason for hiding this comment

NikitaCOEUR Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

NikitaCOEUR Jul 18, 2024

Choose a reason for hiding this comment

suzuki-shunsuke Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

suzuki-shunsuke Jul 18, 2024

Choose a reason for hiding this comment

suzuki-shunsuke commented Jul 19, 2024

suzuki-shunsuke commented Jul 8, 2024 •

edited

Loading

suzuki-shunsuke commented Jul 15, 2024 •

edited

Loading

suzuki-shunsuke commented Jul 15, 2024 •

edited

Loading

suzuki-shunsuke commented Jul 16, 2024 •

edited

Loading

NikitaCOEUR commented Jul 16, 2024 •

edited

Loading

suzuki-shunsuke commented Jul 18, 2024 •

edited

Loading

NikitaCOEUR Jul 18, 2024 •

edited

Loading

suzuki-shunsuke Jul 18, 2024 •

edited

Loading