Allow filtering tags via regex configuration options #2030

lbarcziova · 2023-08-07T06:50:23Z

Introduce 2 configuration options, upstream_tag_include and upstream_tag_exclude to allow filtering the tags. This will be used in the service to allow filtering the propose_downstream runs.

Fixes packit/packit-service#2118

TODO:

Write new tests or update the old ones to cover new functionality.
Update doc-strings where appropriate.
Update or write new documentation in packit/packit.dev.

RELEASE NOTES BEGIN
2 new configuration options for filtering when getting latest upstream release tag were introduced: upstream_tag_include and upstream_tag_exclude. They should contain a Python regex that can be used as an argument in re.match.

RELEASE NOTES END

packit/upstream.py

tests/integration/test_upstream.py

softwarefactory-project-zuul · 2023-08-07T08:26:43Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/cb9f60879e664cc8a8200bf7ef6956c4

✔️ pre-commit SUCCESS in 1m 29s
✔️ packit-tests-rpm SUCCESS in 12m 48s
✔️ packit-tests-pip-deps SUCCESS in 12m 30s
✔️ packit-tests-git-main SUCCESS in 12m 44s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 06s
✔️ packit-tests-git-main-sess-rec SUCCESS in 2m 26s
✔️ reverse-dep-packit-service-tests SUCCESS in 3m 10s

nforro

Since upstream_tag_template is no longer used to filter tags, what happens with projects that rely on it? Should they migrate to upstream_tag_include?

Edit: I mean projects/repos with mixed tag names that use upstream_tag_template for both filtering and extracting version.

lbarcziova · 2023-08-07T08:43:07Z

@nforro that's a good point, I was thinking of opening a PR for the repo that in the past requested the change of utilising upstream_tag_template for filtering (#1885), but not sure whether someone else relies on this/how to find out 🤔 Do you have any suggestions?

softwarefactory-project-zuul · 2023-08-07T11:19:34Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/91883a8fc8d24b21966c7e0fb05c4338

✔️ pre-commit SUCCESS in 1m 26s
✔️ packit-tests-rpm SUCCESS in 13m 37s
✔️ packit-tests-pip-deps SUCCESS in 13m 48s
✔️ packit-tests-git-main SUCCESS in 13m 51s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 01s
✔️ packit-tests-git-main-sess-rec SUCCESS in 2m 18s
✔️ reverse-dep-packit-service-tests SUCCESS in 3m 49s

nforro · 2023-08-07T12:04:13Z

Do you have any suggestions?

Let me try to go through configs using upstream_tag_template and detect mixed tag names usage, I think it shouldn't be too diffcult to do.

lbarcziova · 2023-08-07T12:07:05Z

@nforro for that the configuration-migrations can be used, I can take care of that :)

nforro · 2023-08-07T13:21:35Z

I used configuration-migrations to download the configs and then run a simple script to match the template against all tags:

https://github.com/ansible/pylibssh v{version} 0.4.0
https://github.com/fedora-copr/copr.git python-copr-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git python-copr-common-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-backend-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-dist-git-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-keygen-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-messaging-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-rpmbuild-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-selinux-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-cli-{version} copr-1.1-1
https://github.com/fedora-copr/copr.git copr-frontend-{version} copr-1.1-1
https://github.com/mawww/kakoune v{version} master-0e1afb
https://github.com/sdaps/sdaps v{version} 1.0.0
https://github.com/vcrhonek/hwdata v{version} hwdata-0.146.33.EL-12

That's 5 projects, https://github.com/evgeni/bley is not there because it doesn't actually use upstream_tag_template.

It would probably make sense to check if the "offending" tags are just historical and not a pattern that's still used.

nforro · 2023-08-07T13:23:53Z

@lbarcziova Feel free to check yourself, maybe I missed something.

Introduce 2 configuration options, upstream_tag_include and upstream_tag_exclude to allow filtering the tags. This will be used in the service to allow filtering the propose_downstream runs. Fixes packit/packit-service#2118

lbarcziova · 2023-08-08T10:59:48Z

@nforro thanks! I will check those and open PRs where needed once this lands in production

softwarefactory-project-zuul · 2023-08-08T11:15:04Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/9ab7086010dd47309968fc684c41a684

✔️ pre-commit SUCCESS in 1m 44s
✔️ packit-tests-rpm SUCCESS in 13m 15s
✔️ packit-tests-pip-deps SUCCESS in 13m 28s
✔️ packit-tests-git-main SUCCESS in 14m 06s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 53s
✔️ packit-tests-git-main-sess-rec SUCCESS in 3m 06s
✔️ reverse-dep-packit-service-tests SUCCESS in 4m 11s

softwarefactory-project-zuul · 2023-08-08T11:18:31Z

Build succeeded (gate pipeline).
https://softwarefactory-project.io/zuul/t/packit-service/buildset/d9ff69417f354406abb8bd87eccae2a3

✔️ pre-commit SUCCESS in 1m 28s

We have changed the behaviour of getting latest tags from the repository in Packit (packit/packit#2030) and therefore, for repositories with multiple versioning schemas, it is needed to add these options. For more details, see https://packit.dev/docs/configuration#upstream_tag_include

lbarcziova requested a review from nforro August 7, 2023 06:50

This comment was marked as outdated.

Sign in to view

nforro reviewed Aug 7, 2023

View reviewed changes

packit/upstream.py Outdated Show resolved Hide resolved

nforro reviewed Aug 7, 2023

View reviewed changes

packit/upstream.py Outdated Show resolved Hide resolved

lbarcziova force-pushed the upstream-tag-filter branch from 361452e to de76b69 Compare August 7, 2023 08:06

nforro reviewed Aug 7, 2023

View reviewed changes

packit/upstream.py Outdated Show resolved Hide resolved

lbarcziova force-pushed the upstream-tag-filter branch from de76b69 to 21b516c Compare August 7, 2023 08:10

nforro reviewed Aug 7, 2023

View reviewed changes

tests/integration/test_upstream.py Outdated Show resolved Hide resolved

nforro approved these changes Aug 7, 2023

View reviewed changes

lbarcziova force-pushed the upstream-tag-filter branch from 21b516c to 1b2f0b1 Compare August 7, 2023 11:04

lbarcziova force-pushed the upstream-tag-filter branch from 1b2f0b1 to 2a9a9d8 Compare August 8, 2023 10:14

Allow filtering tags via regex configuration options

0154d12

Introduce 2 configuration options, upstream_tag_include and upstream_tag_exclude to allow filtering the tags. This will be used in the service to allow filtering the propose_downstream runs. Fixes packit/packit-service#2118

This comment was marked as outdated.

Sign in to view

lbarcziova force-pushed the upstream-tag-filter branch from 2a9a9d8 to 0154d12 Compare August 8, 2023 10:58

lbarcziova added the mergeit When set, zuul wil gate and merge the PR. label Aug 8, 2023

softwarefactory-project-zuul bot merged commit 4c510b5 into packit:main Aug 8, 2023
14 of 16 checks passed

lbarcziova deleted the upstream-tag-filter branch August 8, 2023 11:20

lbarcziova mentioned this pull request Aug 14, 2023

Add upstream_tag_include to Packit config fedora-copr/copr#2872

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow filtering tags via regex configuration options #2030

Allow filtering tags via regex configuration options #2030

lbarcziova commented Aug 7, 2023 •

edited

Loading

This comment was marked as outdated.

softwarefactory-project-zuul bot commented Aug 7, 2023

nforro left a comment •

edited

Loading

lbarcziova commented Aug 7, 2023 •

edited

Loading

softwarefactory-project-zuul bot commented Aug 7, 2023

nforro commented Aug 7, 2023

lbarcziova commented Aug 7, 2023

nforro commented Aug 7, 2023 •

edited

Loading

nforro commented Aug 7, 2023

This comment was marked as outdated.

lbarcziova commented Aug 8, 2023 •

edited

Loading

softwarefactory-project-zuul bot commented Aug 8, 2023

softwarefactory-project-zuul bot commented Aug 8, 2023

Allow filtering tags via regex configuration options #2030

Allow filtering tags via regex configuration options #2030

Conversation

lbarcziova commented Aug 7, 2023 • edited Loading

This comment was marked as outdated.

softwarefactory-project-zuul bot commented Aug 7, 2023

nforro left a comment • edited Loading

Choose a reason for hiding this comment

lbarcziova commented Aug 7, 2023 • edited Loading

softwarefactory-project-zuul bot commented Aug 7, 2023

nforro commented Aug 7, 2023

lbarcziova commented Aug 7, 2023

nforro commented Aug 7, 2023 • edited Loading

nforro commented Aug 7, 2023

This comment was marked as outdated.

lbarcziova commented Aug 8, 2023 • edited Loading

softwarefactory-project-zuul bot commented Aug 8, 2023

softwarefactory-project-zuul bot commented Aug 8, 2023

lbarcziova commented Aug 7, 2023 •

edited

Loading

nforro left a comment •

edited

Loading

lbarcziova commented Aug 7, 2023 •

edited

Loading

nforro commented Aug 7, 2023 •

edited

Loading

lbarcziova commented Aug 8, 2023 •

edited

Loading