Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADR-0001: Style guide #191

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@ resolves #
- [ ] I have read [the contributing guide](https://github.com/dbt-labs/actions/blob/main/CONTRIBUTING.md) and understand what's expected of me
- [ ] I have signed the [CLA](https://docs.getdbt.com/docs/contributor-license-agreements)
- [ ] I have run this code in development and it appears to resolve the stated issue
- [ ] This code adheres to the [Style Guide](/docs/arch/adr-0001-style-guide.md)
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Actions and workflows should be self documented. See individual actions for mor
- [Parse Semver Action](parse-semver)
- [Python Package Info Action](py-package-info)
- [Fetch Repository branches](fetch-repo-branches)
- [Fetch COntainer Tags](fetch-container-tags)
- [Fetch Container Tags](fetch-container-tags)

### Workflows

Expand Down Expand Up @@ -51,7 +51,12 @@ Actions and workflows should be self documented. See individual actions for mor
- Want to report a bug or request a feature? Let us know and open [an issue](https://github.com/dbt-labs/actions/issues/new)
- Want to help us build oss actions? Check out the [Contributing Guide](https://github.com/dbt-labs/actions/blob/HEAD/CONTRIBUTING.md)

## ADRs

Would you like to know why things look the way they do?
Do you have questions about where to put something, or how to do something?
Check out our [ADR section](docs/arch/README.md) to see if you can find you answer there.

## Code of Conduct

Everyone interacting in the project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the [dbt Code of Conduct](https://community.getdbt.com/code-of-conduct).

22 changes: 22 additions & 0 deletions docs/arch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Architectural Design Records (ADRs)

For any architectural/engineering decisions we make, we will create an ADR (Architectural Design Record)
to keep track of what decision we made and why. This allows us to refer back to decisions in the future
and see if the reasons we made a choice still holds true. This also allows for others to more easily understand the code.
ADRs will follow this process:

- They will live in the repo, under a directory `docs/arch`
- They will be written in markdown
- They will follow the naming convention `adr-NNNN-<decision-title>.md`
- `NNNN` will just be a counter starting at `0001` and will allow us easily keep the records in chronological order.
- The common sections that each ADR should have are:
- Title
- Context
- Options
- Decision
- Consequences
- Use this article as a reference: [https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions)

## Contents

- [ADR-0001: Style guide](ard-0001-style-guide.md)
76 changes: 76 additions & 0 deletions docs/arch/adr-0001-style-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Style Guide


## Context

We need a style guide for our actions repo. This structure provides the following benefits:
- developers can spend more time on solving the problem
- PR reviews are limited to discussing intent
- format related feedback can point to a collective style guide instead of personal preferences
- consumers know how to find what they're looking for
- developers know where to look when fixing something

## Options

This style guide is a collection of individual options. Instead of providing permutations of these options,
this ADR provides a section for an initial proposal for several style guide topics and a section
that contains the agreed upon formats. This ADR should be updated to remove reference to the initial
proposal prior to merging in order to avoid confusion.

As an alternative, each topic could be its own ADR. Reviewers may request this approach, which will require
breaking this ADR into several ADRs and associated PRs.
Comment on lines +20 to +21
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should split this into workflows and actions. They're different.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we would have one style guide for workflows and another for actions? There's a significant overlap. If we want to distinguish between actions and workflows, but reduce redundancy, can we do one shared style guide, and then an action style guide and workflow style guide that only focus on things specific to actions/workflows?


## Proposal

### Actions
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What types of actions should we prefer - why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion on this. I've been using composite actions since most of the actions I create are pretty small. But some of our actions are more complex and I could see how using python would be easier (like semver parsing). Using python might also be more testable in certain scenarios (again thinking about semver parsing).

- actions live at the repo root so that usage looks like `uses: dbt-labs/actions/<action-name>@<tag>`
- actions live in directories with the format `<platform/tool>/<resource>/<action>`, e.g.
- `./hatch/environment/create`
- `./pypi/release/view`
- actions have a generic scope with a focused context to support reusability
- actions perform a single task on a single object within the context of a platform/tool

### Workflows
- workflows need to live in `.github/workflows`
- we prefer actions to workflows
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree with this. I want to avoid composition of the same 7 actions to accomplish one thing. Having to dig multiple levels down is also frustrating and makes it unclear what's happening.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I struggle with building new workflows off of other workflows for a few reasons:

  • reusable workflows are forced to be their own jobs in the calling workflow
    • I can't embed them in a job
    • I can't pass environment variables to workflows, which is especially a problem when those change by repo (e.g. connection credentials)
    • I need to create a bunch of inputs to pass to them, to get a bunch of outputs from them, to pass those outputs to another job, in many cases to perform one or two additional steps
  • any reusable workflow that needs access to the code requires adding repo and branch/ref as arguments
    • with an action, you do the checkout in the calling workflow and the called action can focus on the task
  • reusable workflows tend to take on too much scope, making them difficult to combine
    • for example, we currently run tests twice during our release for each adapter because both the bumpversion/changelog workflow and build/test workflow runs tests; ideally there is an action for bumpversion, an action for changelog generation, an action for build, and these can all be selected al a carte as needed
    • stated another way, actions should be created to perform a specific task, which should minimize maintenance for a particular action, similar to how python functions should be built

- actions can be inserted into other actions and as part of workflows
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difference here being that workflows can't be part of actions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflow can't be part of actions, and can't be part of other jobs. They need to be stand alone jobs, which is pretty limiting.

- actions can inherit environment variables
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So do workflows?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


### Formatting
- job ids, step ids, and variables are in kebab case
- job names, step names, and description fields follow dbt docs standards (capitalize first word only)
- extra descriptors should be avoided unless required for disambiguation, e.g.
- `version-number` -> `version`
- `archive-name` -> `archive`
- yaml files use a four space tab
- scripts use environment variables in `env` instead of inline substitution like `${{ inputs.value }}`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have a scripts section. When should we write a script vs just doing it inline? Where should scripts live? etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that probably makes sense. To initially respond to the few examples you gave:

  • I am moving more and more towards scripts always; so basically if it's more than a line, make it a script
  • scripts should live alongside the action.yml which calls them, multiple actions should not depend on the same script


### Tools
- formatter: `pretty-format-yaml` @ https://github.com/macisamuele/language-formatters-pre-commit-hooks
- `yamlfmt` @ https://github.com/jumanjihouse/pre-commit-hook-yamlfmt
- `yamlfmt` @ https://github.com/google/yamlfmt (requires Go)
- linter: `check-yaml` @ https://github.com/pre-commit/pre-commit-hooks
- `yamllint` @ https://github.com/adrienverge/yamllint

Alternatives should support `pre-commit`: https://pre-commit.com/hooks.html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatives?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatives to the proposed linters and formatters

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oooh. Should we set up pre-commit in this repo?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes


### Logging
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does logging mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking of debug/summary notifications/etc. as logging. So when we print inputs/outputs to standard out, that's a debug log to me. And when we generate a notification that shows up in the summary section, that's an info/error/warn log.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'll be useful to explicitly distinguish between when to "echo" vs using the GitHub notification that show in the summary.

- the first step of an action logs non-secret inputs as a debug log
- the last step of an action logs non-secret outputs as a debug log
- any step that changes state outside of the scope of the action has an associated info log
- any step that results in predictable failure (e.g. unit tests) has an associated error log
- any step that results in a predictable skip (e.g. version bump not needed) has an associated warning log
- log types are standardized as an action in this repo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You want an action that just does the notify that's built into github?


### Docs
- all actions and workflows are discoverable in the repo `README.md`
- all workflows have a file docstring in the "what, why, when" format

### Versioning
- actions and workflows are not versioned individually; instead, the entire repo is versioned, similar to the `pre-commit` hooks repo
- tags should be used to communicate new functionality
- tagging should follow calver
Comment on lines +70 to +72
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the expectation here that whatever utilizes the workflows works off main or off the calver tag? Would also be useful to link to calver docs and define the specific format we should be using.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and agreed; I'll update this with the link and format.

Folks can use main if they're looking for something to do down the line, or they can use the calver tag and use dependabot to test updates.


## Decision

## Consequences
Empty file added docs/arch/images/.gitkeep
Empty file.
33 changes: 33 additions & 0 deletions docs/arch/templates/default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# { Title }

## Context

{ A brief problem statement. Why do we need to make this decision? }

## Options

- { Option 1 }
- ...

### { Option 1 }

#### Pro's

- { Pro 1 }
- ...

#### Con's

- { Con 1 }
- ...

## Decision

#### Selected: { Option x }

{ Justification }

## Consequences

- [+] { Positive consequence }
- [-] { Negative consequence }