feat: ci: automate the new release process

[galargh]

Closes #12163
Closes #12162

Related Issues

#12010

Proposed Changes

0. Split node and miner releases (node=[lotus], miner=[lotus-miner, lotus-worker]).
1. Publish a new GitHub release (and create an appropriate tag) of node and/or miner whenever a PR which modified version.go (in a way that adds a version that doesn't have a corresponding tag yet) has been merged to either releases or release/vX.Y.Z branch.
2. Run the release workflow in a dry/snapshot mode on PRs that modify version.go (if -rc, then base has to be release/vX.Y.Z, if stable ,then base has to be releases).
3. Run the docker workflow in a dry mode on PRs.

❗ I split homebrew-lotus into 2 formulas (for lotus node - lotus and lotus miner - lotus-miner).

Additional Info

Out of scope for this PR, but worth considering in the future:

1. [DX Streamline] Use the mkreleaselog generated changelog when creating GitHub Releases #12162
2. [DX Streamline] Create draft GitHub releases on PRs modifying version.go #12163

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title is in the form of of <PR type>: <area>: <change being made>
  • example: fix: mempool: Introduce a cache for valid signatures
  • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
  • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
• If the PR affects users (e.g., new feature, bug fix, system requirements change), update the CHANGELOG.md and add details to the UNRELEASED section.
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[github-actions]

All checks have passed

[BigLep]

I think it would be good if this PR also included the accompanying changes to https://github.com/filecoin-project/lotus/blob/master/documentation/misc/RELEASE_ISSUE_TEMPLATE.md so its clear how this affects the "release engineer". Maybe @rjan90 should take this?

[galargh]

I think it would be good if this PR also included the accompanying changes to https://github.com/filecoin-project/lotus/blob/master/documentation/misc/RELEASE_ISSUE_TEMPLATE.md so its clear how this affects the "release engineer". Maybe @rjan90 should take this?

That's a good idea, I'll propose the changes to that file - I think it'll be good way to do the final verification on our mutual understanding of the new process.

[galargh]

I updated the documentation and the accompanying workflows to make sure the automation changes are reflected.

[BigLep]

It looks good to me from my high level read through. We'll want @rvagg to give the approval here though.

In general, and this may just be me, I would like to see less bash for future maintenance regards. That said, I know there are lots of shell scripts in https://github.com/filecoin-project/lotus/tree/master/scripts and I assume that is the convention within github workflow .yml files. Anyways, no insistence on changing this - I just wanted to make sure that it's intentional when we add more shell scripting given it can tend to be more esoteric and less ergonomic than other languages.

[rvagg]

👍 so far

[BigLep]

@galargh : I'll let others chime in here, but a couple of comments or things to be aware of.

1. FYI that we'll be dropping the "releases" branch soon (after the nv23 upgrade) as part of Revised Lotus Node/Client Versioning #12072

2. I believe we should maintain the Homebrew functionality given it is the recommended path on Macs: https://lotus.filecoin.io/lotus/install/macos/

[rvagg]

#12231

[galargh]

1. FYI that we'll be dropping the "releases" branch soon (after the nv23 upgrade) as part of Revised Lotus Client Versioning #12072

Off the top of my head, getting rid of releases will require:

• updating triggers - e.g.

  lotus/.github/workflows/release.yml

  Line 8 in 1b9f812

  - releases

• simplifying this check - https://github.com/filecoin-project/lotus/blob/1b9f8126b1de50472a0701f6bffffc771e78ef1d/.github/workflows/release.yml#L61-62

And that should be about it.

2. I believe we should maintain the Homebrew functionality given it is the recommended path on Macs: https://lotus.filecoin.io/lotus/install/macos/

OK, we'll set it up this will in the homebrew-lotus repository directly.

---

I don't think these two are blocking.

[BigLep]

I assume this is waiting on final approval from @rvagg . I've re-requested review.

[rvagg]

👌 this worked well in #12232 to produce https://github.com/filecoin-project/lotus/releases/tag/v1.27.2-rc2

I'm happy with this if @rjan90 is.

[rvagg]

Some issues encountered when taking v1.27.2 all the way that will need to be addressed here: #12249 (comment)

[galargh]

This required a further merge of release/v1.27.2 into the releases branch to finalise #12250, which is different from RC builds. This is as designed but needs to change for v1.28.0 forward because we are not using the releases branch for that.

This was expected as per #12096 (comment). Updated now in c1d8638

This created miner builds and a miner release: https://github.com/filecoin-project/lotus/releases/tag/untagged-086fb5cb1c887e1c1658, although the release page is mostly empty except for some titles and the build assets. I cancelled the miner release step before it did its thing so we haven't pushed that: https://github.com/filecoin-project/lotus/actions/runs/9972453588/job/27556032962

This was expected, too, since miner/v1.27.1 hasn't been released before and v1.27.1 is the miner version in the releases branch. The notes for miner releases are sourced from changelog.

The release page it posted had two titles and two contributor tables; one taken from the CHANGELOG and one generated. The contributors table that was generated was more correct although the title wasn't an h2 (##), I forgot to redo it. But I edited the release page after the draft was created, but when it did the final release it overwrote my edits so I have to edit it again. It would be good if the draft could be used to get it right before it's pushed to the wild since some people get the original via email or other notification where the edits won't be seen after publish.

Removed secondary title in 55d83f0

The release notes are changelog entry + mkreleaselog output. You no longer have to paste the mkreleaselog output in changelog directly.

Added draft notes preservation in 55d83f0

[rvagg]

This was expected, too, since miner/v1.27.1 hasn't been released before and v1.27.1 is the miner version in the releases branch. The notes for miner releases are sourced from changelog.

Oh .. we set the miner version to be something without -dev: 8e7cd7d, we probably should leave it as -dev so we don't get unwanted releases.

[BigLep]

@galargh: are the next steps clear here so we can get this merged?

[BigLep]

@rvagg : per slack, @galargh believes this is good to go. Will you please merge?

[rvagg]

Has conflicts that need resolving. @galargh I'm happy for you to squash merge this once you're done with the conflicts.

[galargh]

I don't see any outstanding conflicts. Is it with some other branch than the target (master)? Let me know, happy to update as needed.

[rvagg]

hm, no, I'm sure there were conflicts when I posted that message! Looks good to go now.

[galargh]

Might have been some glitch with GitHub UI; thank you for merging, let me know if there are any issues with it and I'll jump straight on it.