gocollector tests: Adjust CI to tell us when generated files were manually changed + refactor?

[ArthurSens]

While working on #1559, we noticed that the generated files for the go collector can be manually changed and CI won't block the merge.

Let's investigate this and adjust :)

[bwplotka]

Yup, CI is NOT running this generative script.

We only check if metrics given by the runtime is what we expect in those generated _test files and then complain with instruction to potentially rerun that script. This helps us to figure out what metrics changed across Go version. That's the main goal of those files.

You might be right some CI check that would rerun this script and point mismatch on TOP of our existing test would let us know if someone shoveled some custom manual code to those files.

Do you mind adding clear acceptance criteria to description?

[bwplotka]

BTW.. we have two of those scripts 🙈

https://github.com/prometheus/client_golang/blob/main/prometheus/gen_go_collector_metrics_set.go
https://github.com/prometheus/client_golang/blob/main/prometheus/collectors/gen_go_collector_set.go

[bwplotka]

This might need proper refactor e.g. have one generated set in internals and separate logic to filter those for test purposes (it's just regex)

[bwplotka]

Ideally I would also have only one script that generates both package files 🤔

[bwplotka]

Acceptance Criteria

• One script to generate test helpers for two places, we can move it to scripts dir or so.
• Make sure our CI job for periodic script run and check actually works for BOTH files - it looks like it ran only one chore: Update metrics for new Go version #1597 recently
• Fix conflicts (visible in chore: Update metrics for new Go version #1597) as we added manual changes to generated files in Add default Go runtime metrics for /gc/gogc:percent, /gc/gomemlimit:bytes, /sched/gomaxprocs:threads #1559 - those manual changed have to be either moved somewhere else or moved to generated content. This might need some test refactor
• Have another CI job that checks for manual changes on those files.