Tracking Issue for Stale Bot

[dwijnand]

We are evaluating the use of the Stale bot, and this is the tracking issue to gather feedback on it.

[dwijnand]

Initial setup: #6020
Messaging tweak: #6030

[gnzlbg]

I find this extremely annoying.

[retep998]

I really really really hate this stale bot.

[SimonSapin]

I disagree with the very premise of this bot. An issue without recent activity often means that it hasn’t been prioritized, that doesn’t make it less of an issue.

What is the problem that this is trying to solve? PR #6020 that set this up gives no reasoning or discussion at all.

[gnzlbg]

What is the problem that this is trying to solve?

There does not seem to be enough manpower to triage cargo issues, so this bot is basically pinging every issue to force the people that participated in them to triage them themselves.

This will probably result in issues being closed if those who participated in them got hit by a bus - just because nobody in the issue answers this bot does not imply that the problem the issue describes has been fixed :/

cc @alexcrichton

[dwijnand]

So I proposed using this bot. I realise this is a contentious issue (I believe Aaron mentioned it's been discussed in the past in the context of the Rust issue tracker).

My motivation is that in trying to help out resolving issues in the Cargo project I found myself often bumping into issues that just aren't ready for an implementation, often because its isn't clear what the issue is or what solution is desired, making it effectively un-actionable.

So I proposed using this bot to help close off some issues so that the issues that have the greatest interest can stay open, and those that are more niche don't clog up the backlog.

Now, I too disagreed with the premise of this bot, but have since changed my mind.

A closed issue doesn't mean the issue isn't valid, and a closed issue doesn't mean the issue was resolved. Simply, there isn't sufficient interest in the issue to keep it open at this time. However issues can also be re-opened and, indeed, the team has no problem with that.

Alex kindly accepted trialing this bot. I personally think that using this bot can be beneficial to a project. But every project and community is different, and this is just an evaluation. If you have any suggestions or recommendations for its usage, I/we'd be interested to hear them.

[retep998]

Issues should not be closed just because they're "niche" or low priority, especially when "niche" often is just whatever the people in charge don't care enough about. Issues should only be closed when they've been resolved. There are better ways to prioritize issues, whether via priority labels or meta issues or github projects or external lists somewhere.

There is a need to ensure old issues get triaged, and there should definitely be some sort of thing to automatically direct some sort of triage team at a rotating list of inactive issues to see what can be done to get them moving. Automatically closing issues is not triaging, it's just sweeping issues under the rug, and it definitely won't solve issues with a lack of manpower.

[SimonSapin]

My motivation is that in trying to help out resolving issues in the Cargo project

If the scenario we want to improve is that of a contributor searching for something to work on, I think that a positive signal like the E-easy label is much better suited.

its isn't clear what the issue is or what solution is desired, making it effectively un-actionable.

Just because an issue isn’t immediately actionable doesn’t mean that there isn’t a real problem that should be resolved.

A closed issue doesn't mean the issue isn't valid, and a closed issue doesn't mean the issue was resolved.

I suppose this is where we fundamentally disagree. I view closing an issue as a statement that no further action will be pursued: either the issue is resolved, or we made a decision not to resolve it (even if that decision can be reversed later).

clog up the backlog

How do inactive issues “clog up” anything? What does that mean? Why is that a problem?

GitHub allows sorting issues by most recently updated. This way you can avoid looking at them, without closing them.

[elwl]

Perhaps automatically add a tag for inactive after x number of days of no activity on the issue (and automatically remove the tag on activity). It certainly shouldn't be closed though, closing an issue implies either it's resolved, or there's no intention for it to be resolved.

[gnzlbg]

An alternative would be to just use P-low, P-medium, P-high labels to prioritize things. Everything that doesn't have a P-label, can be interpreted as P-lowest.

[alexcrichton]

Thanks all for taking the time to provide some feedback on this, it's definitely appreciated! We discussed this during the @rust-lang/cargo triage meeting yesterday, and I'm going to try to do my best to summarize the result of the discussion.

The main aspect that we concluded was that we view it as quite valuable to, in an automated fashion, poke issues to try to keep them moving forward. This is a great way for the team to be reminded about older issues (as well as contributors). This not only provides an opportunity for someone to realize that the issue has been fixed by something else but it also provides a great time to refresh the context to make sure the bug is up to date.

There was not consensus amongst the team about the usefulness of automatically closing issues if they had been stale for awhile. We all agreed, however, that many of the points brought up here were agree that we don't want to happen. For example we do not want to portray the message that if an issue is automatically closed that it will be forever closed. Furthermore we don't want to irritate ourselves or contributors by constantly fighting to keep a legitimate issue open.

With all that in mind, and in addition with the experience we've learned over the past weekend, we concluded that this is the steps we'd like to take at this time:

• We've updated the bot's configuration to slow down even more. Although 24 issues per day is as slow as the bot can go, we felt it was still too fast. We've increased the time it takes for an issue to be considered as "stale" (to three years) to only have a handful of issues be considered for being stale.
• We've also updated the amount of time given until an issue is automatically closed, now to 30 days. We're in no rush to close a stale issue, and we want to make sure that if an issue is closed that it truly is stale with very little interest!
• We're looking for feedback on wording of the message to clearly imply that the main purpose is poking the issue to help it move forward. We don't want the focus to be that this is closing and it's a race against the clock.
• We won't decrease the amount of time to the first poke by the bot until we can figure out how to execute at a slower rate than 24/day. This may require changes to the bot upstream, however.
• We'd love to get some feedback on labels that turn off the bot. For example right now all issues tagged as a feature request will not be poked by the bot. Are there other labels to add too?

Others from the @rust-lang/cargo team can feel free to chime in here too!

[SimonSapin]

we don't want to irritate ourselves or contributors by constantly fighting to keep a legitimate issue open.

Maybe the lower frequency will make this less irritating, but it’ll keep happening.

[alexcrichton]

@SimonSapin that's true, yes, but if truly nothing happens on an issue for literally years on end I think asking someone to say (like once a year) that the issue still exists is quite reasonable. If an issue has gone multiple years with only a bot and one person poking it then it's highly likely that it's entirely un-actionable its current form and needs to change somehow, and leaving it simply sit and collect dust won't actually do that.

[durka]

• We've increased the time it takes for an issue to be considered as "stale" (to three years)
• We won't decrease the amount of time to the first poke by the bot

Sorry, I think I'm misinterpreting something but these seem contradictory... to clarify, how old will an issue get before the bot first pokes it?

---

Regarding tags that disable the bot, it seems to me that almost any tag should do it. That is, if a team member comes along to add tags and says "gee, this issue looks really unclear/unactionable/unreproducible", I'd imagine it should stay untagged or get a C-needs-details-plz tag right from the get-go. But yeah, I realize that's kind of idealistic and it's still quite possible for issues to become outdated.

I still think there should be a more human touch before closing issues, like maybe the bot can assign a random person to follow up like highfive does (I'd be on that ping list if you need volunteers).

[alexcrichton]

@durka the exact amount of time is up in the air, currently it's set at three years. We won't make that shorter, though, until we can get the rate limits under control.

[dwijnand]

• We won't decrease the amount of time to the first poke by the bot until we can figure out how to execute at a slower rate than 24/day. This may require changes to the bot upstream, however.

I opened probot/stale#165 to track that. My initial attempt to fix that wasn't successful.

[dwijnand]

Given we can't get stale bot to operate as we'd like (poke up N older-then-X-period tickets a day) I'd say let's tear it down and close this ticket.

I think we should keep the tickets closed by the bot marked as stale.

@rust-lang/cargo WDYT?

[alexcrichton]

Sounds good to me, thanks for investigating though!