[DX Streamline] Lotus node-only release process

[rvagg]

Current workflow runs make deps lotus lotus-miner lotus-worker and GoReleaser config bundles the 3 binaries into a single tgz for uploading to the release page.

We need a streamlined way to produce lotus-only releases. Possibly using some variation of the IPDX version.json tooling (e.g.) such that it simplifies some or all of the steps outlined in the release flow checklist.

e.g. A pull request changing build/version.go will trigger other actions in that release flow, including setting up a draft release page, generating a basic changelog, running gen and docsgen-cli etc. When the PR is closed, it's tagged and a release is published. Somewhere in there, a flag that says "lotus-only", or the inverse, should indicate whether to include the additional components.

e.g. Alternatively, just work within the current process but add optionality to GoReleaser and release.yml that can switch between two modes with a simple flag from somewhere.

[rvagg]

Outstanding question: all components use api/version.go to determine their build version, should we separate this out to have the daemon using a different version number. If we tie them together, even if we only release a daemon binary, that tag would give you a compile of lotus-miner that has the same version, yet it may not be something we really want people to compile and use because. Maybe we need a separate file for both.

[galargh]

We need a streamlined way to produce lotus-only releases. Possibly using some variation of the IPDX version.json tooling (filecoin-project/lassie#471) such that it simplifies some or all of the steps outlined in the release flow checklist.

Yes, using something like a version.json file would definitely help. The big advantage of it is that it's easy to parse and use the information on the version in other parts of the release flow. In particular, it would be possible for us to get the version information from version.json in build/version.go.

Alternatively, we could use something like a changelog driven release process. We do that successfully for releasing all the GitHub Actions that we maintain. In that setup, the information about the current version is retrieved from the CHANGELOG.md. Markdowns are a little bit trickier to parse, but the advantage is that you only have to update the file that you're looking at anyway during the release. We did a write-up on that not too long ago - https://www.ipdx.co/articles/guides-articles/ipdx-recipe-how-we-saved-time-and-mental-overhead-with-changelog-driven-releases

With lotus and lotus-miner following separate release cycle, we could store multiple entries in the version.json file or maintain multiple CHANGELOG.mds (technically, we could do it from one too but it seems a little confusing to me).

The other question that I think needs answering is how to handle tagging with lotus and lotus-miner following separate release cycles but still live in the same repository. I think it'd be quite dangerous not to have both released for each tag. One option that comes to mind to work around this is to use prefix miner- for the miner releases and either no prefix for lotus or a different one.

[rvagg]

Continuing the discussion from #12035 (comment)

1. I agree that we should have separate tags for miner and node, just prefixing miner with miner- and leaving the node releases as plain semver strings seems fine to me.
2. Being able to release both in a single PR if we ever decide it's necessary is fine too as long as it doesn't add too much complication to the process - if we can use matrixes to achieve this simply then that's great, my concern is more about how much complexity this adds to the config (and how much time it takes to achieve!).

So, I think we're down to a remaining question of how to trigger a release: I've proposed version.go as the signalling mechanism in #12035 - open a PR changing version and it goes through the steps to prepare a release. The alternative proposed here is updating the CHANGELOG, which I imagine will also bump version.go automatically if it detects a new version header. @rjan90 I think this might be a question for you - do you have any thoughts or preferences for how to trigger a release? If you got a branch all ready for a release and just opened a final PR, or added a final commit to a PR, what would you like it to be to get CI to recognise that you want to start the release process and then upon merge have it perform the actual release steps?

[rjan90]

@rjan90 I think this might be a question for you - do you have any thoughts or preferences for how to trigger a release? If you got a branch all ready for a release and just opened a final PR, or added a final commit to a PR, what would you like it to be to get CI to recognise that you want to start the release process and then upon merge have it perform the actual release steps?

I believe the proposed mechanism of signaling a release through a change in version.go makes the most sense. It is straightforward and simple, making it an effective way to trigger the release process.

[jennijuju]

I agree that we should have separate tags for miner and node, just prefixing miner with miner- and leaving the node releases as plain semver strings seems fine to me.

SGTM. & lotus version and miner version should be decoupled as well

[galargh]

Sounds great :) We'll get started on setting up the automatic release triggering based on changes to version.go. We'll base it off of #12035. We'll first publish the setup as a draft so that we can discuss it further and make sure it fits within the new release process.

[rjan90]

I think the two remaining tasks, before this can be closed is:

• Get the automatic release triggering based on changes to version.go PR up
  - [ ] Figure out how to seperate Miner and Node Changelog entries.
• Publish a succsseful Lotus release with the automatic release triggering

[galargh]

I think the two remaining tasks, before this can be closed is:

• Get the automatic release triggering based on changes to version.go PR up
• Publish a succsseful Lotus release with the automatic release triggering

As confirmed on Slack, the ETA on preparing the setup is next week the latest.

[BigLep]

@rjan90 : what's remaining before we can close this? Should we maybe move the checklist in #12010 (comment) up to the issue description? I assume "Publish a succsseful Lotus release with the automatic release triggering" should become two tasks: one for "Lotus Node" and one for "Lotus Miner".

[galargh]

I believe both were accomplished now, Lotus Node and Lotus Miner 🎉

[rjan90]

Closing this ticket as completed.