FY2023 Q4

[chadwhitacre]

← Q3 · Q4 · Q1 →

---

Build

• ✅ Engagement - Migrate Customer Feedback flows to public GitHub #42
  • PMs are now active in discussions.
  • CSM/SEs are now active in issues.
  • Vanguard: COPS Now Sends Feedback to GitHub
• ✅ Engagement - Get routing and triage above 90% #57
  • On the last day of the quarter, recently routed/triaged are at 92% and 94% (dashboard)
  • Quarterly impact from notification changes should arrive in Q1.
  • More changes slated for Q1.
  • Vanguard: Moar regular triage notifications!
• ⛔️ Self-hosted - Reduce chaos with better end-to-end testing  #55
  • Tests are standardized across all relevant repos.
  • We have expanded test coverage.

Maintain

• ✅ Self-hosted - Releases
  • Release 22.11.0 self-hosted#1754
  • Release 22.12.0 self-hosted#1813
  • Release 23.1.0 self-hosted#1868
• ✅ TTR/TTT
  • TTR - 90% is a big improvement over 75% in Q3
  • TTT - 84% is basically our best ever (86% in FY2022, Q2 had a 4-day target vs. 2-day)
  • took a hit over the holidays, will be factored out in future years
• ⚠️ Engagement 🥔s
  • Lost track of this over the holidays, dusted it off in the new year.
  • Confirmed it is valuable for SDK teams especially (most active segment by far).
• ⛔️ Quick Wins
  • Haven't kept up.
  • Probably should be part of a larger education program.

No

• Self-hosted - Fix backup/restore flow sentry#36868
• turn FOSSA back on - we're waiting for them to ship improvements before we revisit
• Build out an attribution page.
• Add segments to 401 viz to better target 🥔s.

[chadwhitacre]

Epics:

• Reduce chaos with better end-to-end testing  #55
• Get routing and triage above 90% #57

[chadwhitacre]

• Probably can move tests for backend changes to daily since we publish nightly images
• Suggested plan: write RFC for moving to daily test runs, try it out for a while (see if it is too burdensome for us to triage if there is a test failure)

As @BYK says this is based on a faulty assumption that nightly means nightly. 😅 In #55 we're focusing on speed of tests, if we can get down to reasonable then no need to async.

[chadwhitacre]

Biweekly update:

• feedback - decision taken on Oct 28 (w/ David, CDV, Katie, Neil) to implement a GitHub => Jira connector; working with Eran on EPD side; vanguard post pushed back ~a month to Dec 1
• triage automations cleanup - getting underway
• e2e testing cleanup - getting underway

[chadwhitacre]

Biweekly update:

• feedback - Focus of work is a big Vanguard post. GTM asks are pretty well firmed up, working through Unito rollout. EPD asks are coming along behind, evaluating "Build in Public" as a framework for us.
• triage cleanup - Quick win for Support and tech foundation being laid. Wider value arriving in December and into January.
• e2e testing - Tests standardized in 3 of 4 repos. Test improvements to follow after that.

[chadwhitacre]

Biweekly Update

1. 🟢 Customer Feedback—Improve communication between customers, GTM, and EPD by migrating from private Jira to public GitHub.
   1. I calved off a PM/PMM ship from the big Vanguard post.
   2. Post is focused exclusively on GTM now. High alignment across teams (CSM/SE/CS).
   3. Last big blocker is purchasing Unito, very close and then it should all be downhill.
2. 🟢 Triage Cleanup—Meet our routing and triage time targets by improving our GitHub notifications.
   1. The quick Support win is making a big difference, validating our overall approach.
   2. We shipped foundation work for EPD teams.
   3. We're on to visible improvements.
   4. We defined a stretch goal that will probably turn into next quarter's project.
3. 🟨 E2e Testing—Standardize our self-hosted e2e testing to catch errors without slowing down other teams.
   1. Limited progress, still at 3 of 4 repos standardized.
   2. Fourth underway.

(internal)

[chadwhitacre]

Biweekly Update

1. 🟢 Customer Feedback—Improve communication between customers, GTM, and EPD by migrating from private Jira to public GitHub.
   1. Unito is purchased and deployed and I had a kick-off with 25 CSMs and SEs today!
   2. Post cannibalized for CSM/SE docs.
   3. Onboarding for CSM/SE will continue for a couple weeks, aiming for a shipped by January 15.
   4. Next up are some process improvements for Customer Support and that will round out the project for the quarter.
2. 🟢 Triage Cleanup—Meet our routing and triage time targets by improving our GitHub notifications.
   1. Okay so turns out they weren't actually visible improvements, but we are now computing TTR/TTT in hours instead of days, and we're accounting for company holidays (per office).
   2. We have another sprint worth of infra, and then we can get to the big visible change, which is much better Slack notifications for EPD teams. Aiming to ship the week of January 9 with post to follow.
3. 🟨 E2e Testing—Standardize our self-hosted e2e testing to catch errors without slowing down other teams.
   1. Limited progress. Fourth repo brought under standardized e2e testing, but ...
   2. ... we got bogged down sorting out pushing to Docker Hub.
   3. Aiming to land Standardize e2e testing self-hosted#1756 this week.

(Notion, email)

[chadwhitacre]

Biweekly Update

1. 🟢 Customer Feedback—Improve communication between customers, GTM, and EPD by migrating from private Jira to public GitHub.
   1. PM in discussions is taking root, good engagement re: crons especially.
   2. CSMs and SEs are off and running. Will continue to coach and refine.
   3. Customer Support is on the new feedback process, working on leveling up in related areas.
   4. Still aiming for a shipped by January 15.
2. 🟢 Triage Cleanup—Meet our routing and triage time targets by improving our GitHub notifications.
   1. Infra done.
   2. Now for visible improvements! Watch for much better Slack notifications for EPD teams.
   3. Aiming to ship the week of January 9 with post to follow.
3. 🟢 E2e Testing—Standardize our self-hosted e2e testing to catch errors without slowing down other teams.
   1. Landed e2e test standardization across repos, so calling this green again.
   2. Working on improvements to the now-standardized test suite.

(Notion, email)

[chadwhitacre]

Biweekly Update

1. 🟢 Customer Feedback—Improve communication between customers, GTM, and EPD by migrating our feedback workflow from private Jira to public GitHub.
   1. All relevant teams are on the new process! 💃
   2. Here are outbound (PM) and inbound (COPS) conversations.
   3. Decided to make a small improvement before declaring victory.
   4. Aiming for a shipped by January 25.
2. 🟢 Triage Cleanup—Meet our routing and triage time targets by improving notifications.
   1. New Slack notifications are live! 💃
   2. We are monitoring the triage and routing numbers.
   3. Working on cleanups and a shipped this week or next.
3. 🟨 E2e Testing—Improve self-hosted e2e testing to catch errors without slowing down other teams.
   1. Back to yellow.
   2. Worked through a CI incident last week.
   3. One new test added.
   4. Two more new tests remaining in scope.

(Notion, email)

[chadwhitacre]

Biweekly Update

1. 🟢 Customer Feedback—Improve communication between customers, GTM, and EPD by migrating our feedback workflow from private Jira to public GitHub.
   1. Small improvement has been made. We now have GTM separated out on the dashboard.
   2. Vanguard post is up.
2. 🟢 Triage Cleanup—Meet our routing and triage time targets by improving notifications.
   1. We're up over 90% for both TTR and TTT! Woo-hoo! 💃
   2. Vanguard post is up.
   3. Landed some additional cleanups.
3. ⚠️ E2e Testing—Improve self-hosted e2e testing to catch errors without slowing down other teams.
   1. Still at yellow.
   2. One new test added since last time.
   3. One more new test remaining in scope ... miiiiiiight land today? 🐭

(Notion, email)

[chadwhitacre]

New quarter!