Propose Developer Experience project

[tsloughter]

No description provided.

[svrnm]

Thanks for proposing this project @tsloughter! Some feedback to the proposal

[dmathieu]

I am interested in joining this project.

[yurishkuro]

I recommend the name Usability, not Convenience.

[tsloughter]

I'm good with a name change if people prefer "usability". I was also considering "developer experience".

[austinlparker]

Overall, I support this!

edit: I like 'Developer Experience' too.

[tsloughter]

@yurishkuro what do you think of "developer experience"?

[yurishkuro]

yeah, sounds good

[theletterf]

This is necessary. I wonder, though, about the following:

• Why 3 problems only? Why not generating a ranked list of issues?
• Surveys can be tricky. What would the sample be? The PoV of community members might be biased by years of experience. How to reach beginners and new users?
• Have we considered running UX research interviews?

[tsloughter]

@theletterf the plan is a ranked list where we pick the top 3. The 3 is just to give the project a goal to reach rather than an open ended, "fix this list". After the 3 the group can decide whether to propose continuing as a full SIG or disbanding and waiting to regroup when there are resources available again to take on more issues.

How to reach the right users will be worked on with th End User SIG.

UX research interviews? Would that be like a user trying to instrument an application for the first time live with an interviewer watching?

[julianocosta89]

+1 for "Developer Experience"and I'd like to be involved in this SIG.
Thanks @tsloughter for bringing it back to life!

[martinjt]

More than happy to be involved.

One note from skimming through. I believe we should stay away from making developer experience functions, methods etc. Consistent across all languages.

The key issues I see right now are when we try to make something work across all languages the same when there are massive differences between the idiomatic style and flow of different languages and ecosystems.

I think what we need to do is work out how we provide spec flexibility to allow them, while ensuring standards.

[tsloughter]

I've added a section about the use of first talking to language SIG's to get their initial feedback.

[tsloughter]

As for the naming, DevEx/DX is a pretty common and well known term I believe. Enough so to be abbreviated often. Looks like "contributor experience" is as well. So I'd argue they should both remain with the same names.

We could consider something like, "End User Experience", but I think that is too broad while we are focused on developers using the API/SDK.

[samsp-msft]

Do we have infrastructure from the CNCF etc for running surveys. devdiv@Microsoft uses SurveyMonkey as they have good infrastructure and can have org policies for PII management etc. We typically use surveys for two purposes:

• statistical & sentiment analysis. Verbatim responses are useful when categorized to understand high level problem areas
• Collecting contact details for further follow up in zoom/team calls to do a deeper interview as to the customer experience.

The main challenge is getting the survey in-front of the target audience's nose. Github doesn't have good infra for notifying users of a survey (such as having a banner). Pinned discussions depend on the user visiting that area, same for issues.

Second is going to be managing PII, this is an open group, but most participants won't want their information to be spread over the internet - any PII needs to be sandboxed and only used for the stated purpose, and have an expiry.

[austinlparker]

Do we have infrastructure from the CNCF etc for running surveys. devdiv@Microsoft uses SurveyMonkey as they have good infrastructure and can have org policies for PII management etc. We typically use surveys for two purposes:

• statistical & sentiment analysis. Verbatim responses are useful when categorized to understand high level problem areas
• Collecting contact details for further follow up in zoom/team calls to do a deeper interview as to the customer experience.

The main challenge is getting the survey in-front of the target audience's nose. Github doesn't have good infra for notifying users of a survey (such as having a banner). Pinned discussions depend on the user visiting that area, same for issues.

Second is going to be managing PII, this is an open group, but most participants won't want their information to be spread over the internet - any PII needs to be sandboxed and only used for the stated purpose, and have an expiry.

We usually just use Google Forms. IIRC CNCF does use SurveyMonkey for stuff, there's probably a way we can get on it. To be quite honest, the biggest challenge the project faces in terms of surveys is (as you intuit) that we do not have a great way to get the survey in front of users other than social media and Slack messages.

That said, we can work with our vendor community to help broadcast/promulgate surveys and other feedback mechanisms, which we'll probably end up doing in this case. My generic preference is that rather than doing time-limited surveys, we build out better continuous reporting mechanisms, but that's kinda neither here nor there.

Regarding PII, we don't really wind up with PII issues because we don't collect any (I'm not sure we even collect e-mails on our existing surveys).

[austinlparker]

What I am interested in is a group dedicated to the experience for Application & Library developers who are consuming OpenTelemetry and using them in their applications. That is a much larger and different audience from those contributing to OpenTelemetry or writing tools such as an APM to consume and display data collected by OpenTelemetry.

While I think we are talking about similar things here, I also feel like perhaps we are not. This SIG -- at least, initially -- is designed to focus on the needs of instrumentors; Developers who are writing OpenTelemetry code in their applications and libraries. While the other things you mention are important, and are part of DevEx, I think it's important for us to keep the initial scope of this SIG constrained.

If this SIG is successful in these initial goals, then it would seem prudent to expand to other topics, but given the amount of backlog the project currently suffers I believe we need to find some 'quick wins' as it were to address the main ergonomic pain points of instrumentors.

[tsloughter]

+1 to everything @austinlparker said.

I completely agree about the importance of the other parts of developer experience, and honestly have looked Aspire as something I want to build for Erlang/Elixir -- like I started doing with Erleans to copy another Microsoft project :) -- but I wanted to purposely keep the scope of this project small to focus the group and not spread us too thin.

Then, things can change in a future DevEx project (or SIG) after the roaring success of this initial iteration :).

Whether to limit the scope of surveying as well can be left to discussion within the project. Maybe we'll want to still surface other areas of improvement in the initial report but limit our focus of work on the ones related to the spec.

As for discussion of how to perform the surveying I think we can leave that to after we've gotten this merged and take it to the project group?

[danielgblanco]

Thanks for the proposal!

[danielgblanco]

As for discussion of how to perform the surveying I think we can leave that to after we've gotten this merged and take it to the project group?

I agree. The End-User SIG has some basic guidance on how surveys are usually conducted https://github.com/open-telemetry/sig-end-user/tree/main/end-user-surveys.

I also think that scoping this SIG to a few actionable items makes a lot of sense. After this initial effort the SIG can re-evaluate if these efforts should continue on a more long-term basis in its current form (like a permanent SIG) or if there's a better way to tackle those issues in a different way, perhaps involving a combination/collaboration between End-User SIG and implementation SIGs directly.

[lmolkova]

I'm in complete agreement with the proposal, but I think we need to have consensus about the name. We have a proposal for a "contributor experience" SIG, and it will likely be confusing to have a "developer experience" one as well, especially as a good number of contributors are developers.

throwing in some alternative naming suggestions:

• Instrumentation Development Experience
• Instrumentation Experience
• Instrumentation Convenience

[martinjt]

One of the externally facing goals of this, at least in my eyes, is to give the users of OpenTelemetry confidence that we're making it better for them.

While I can understand the desire to try and have very well segregated and defined names internally, I think it will be detrimental to that goal.

We could say "developer experience is an overarching project covering contributors to the project as well as end users", however, I think that would soften the message too much.

Developer Experience is a term that, external to OSS Contributors, is widely known as making the tool easier to use for the people use it.

In my eyes, we should absolutely stick with this being referred to as the "Developer Experience" project, and not try and segment/qualify it. That distinction can be made in the first paragraph and have the same effect, without diminishing the impact on the other experience projects.

[austinlparker]

If anyone has very strong opinions that this project should not be named Developer Experience, I would appreciate their feedback as soon as reasonably possible so that we can close on this and move forward.

[jpkrohling]

I need to catch up with this discussion, but I know that both @tedsuo and @svrnm expressed concerns about the name last week. I like @lmolkova's suggestions, having a slight preference for Instrumentation Development Experience, followed by Instrumentation Experience.

[austinlparker]

I need to catch up with this discussion, but I know that both @tedsuo and @svrnm expressed concerns about the name last week. I like @lmolkova's suggestions, having a slight preference for Instrumentation Development Experience, followed by Instrumentation Experience.

Could @tedsuo and @svrnm post their concerns in this PR so we're not playing telephone then?

[svrnm]

I need to catch up with this discussion, but I know that both @tedsuo and @svrnm expressed concerns about the name last week. I like @lmolkova's suggestions, having a slight preference for Instrumentation Development Experience, followed by Instrumentation Experience.

Could @tedsuo and @svrnm post their concerns in this PR so we're not playing telephone then?

Apologies for not coming back on this issue earlier, I was at KCD Munich and had some other things to follow up with.

I do not have a strong concern with it, I just want to make sure that "Developer Experience" how I understand it and what the SIG plans to do is congruent.

A good definition I found for Developer Experience comes from this github blog post:

DevEx refers to the systems, technology, process, and culture that influence the effectiveness of software development. It looks at all components of a developer’s ecosystem—from environment to workflows to tools—and asks how they are contributing to developer productivity, satisfaction, and operational impact.

Another one from the Microsoft Engineering Fundamentals Playbook:

Developer experience refers to how easy or difficult it is for a developer to perform essential tasks needed to implement a change. A positive developer experience would mean these tasks are relatively easy for the team (see measures below).

The essential tasks are identified below.

Build - Verify that changes are free of syntax error and compile.
Test - Verify that all automated tests pass.
Start - Launch end-to-end to simulate execution in a deployed environment.
Debug - Attach debugger to started solution, set breakpoints, step through code, and inspect variables.

If effort is invested to make these activities as easy as possible, the returns on that effort will increase the longer the
project runs, and the larger the team is.

If I read the proposal the focus is on the API/SDK "easy of use" or providing convenience functions to use them:

The first deliverable will be the collection of experience in dealing with
developer experience issues by each existing language SIG. This means not only
additions to the API/SDK or libraries developed to enhance the experience for
users but any that may be planned or are being thought about because of a
frequent request from their users.

Based on what people commonly understand under "tools" for developer experience I see the following issue:

Tools is much bigger than what the language APIs/SDKs are doing, and also from the definitions I found and the understand I had, tools people have in mind are IDEs, AI assistance, Automation Tooling for Tests, Helpers for Debugging, maybe even the chair they are sitting on etc. Based on this comment by @samsp-msft and the answer by @austinlparker this seems not to be in scope (initially) and there is the desire to keep the scope small.

So the SIG needs to ask themselves if/how to manage expectations if people assume that all of this is part of what they are going to do? Is this SIG going to look into making OpenTelemetry easier to build, test, start and debug?

If the answers to those questions is "Yes" I have no further objections.

[austinlparker]

So the SIG needs to ask themselves if/how to manage expectations if people assume that all of this is part of what they are going to do? Is this SIG going to look into making OpenTelemetry easier to build, test, start and debug?

If the answers to those questions is "Yes" I have no further objections.

If those are what's needed to make the developer experience better, sure? Better APIs isn't just "there should be declarative span creation sugar", it could also be "APIs should be easier to test" or "SDK configuration should be easier"

The point of calling this Developer Experience is in the spirit of keeping our options open. If we said "oh this is just instrumentation convenience" and there's overwhelming feedback that the biggest problem is, like, SDK initialization or testing, then we'd be artificially limiting ourselves.

Now, do I think that those will be the biggest pain points or the first thing being tackled? No, I suspect that the biggest DevEx problems we have right now are around the actual instrumentation API, but I don't want to presume things that aren't backed by data.

[svrnm]

So the SIG needs to ask themselves if/how to manage expectations if people assume that all of this is part of what they are going to do? Is this SIG going to look into making OpenTelemetry easier to build, test, start and debug?
If the answers to those questions is "Yes" I have no further objections.

If those are what's needed to make the developer experience better, sure? Better APIs isn't just "there should be declarative span creation sugar", it could also be "APIs should be easier to test" or "SDK configuration should be easier"

The point of calling this Developer Experience is in the spirit of keeping our options open. If we said "oh this is just instrumentation convenience" and there's overwhelming feedback that the biggest problem is, like, SDK initialization or testing, then we'd be artificially limiting ourselves.

Now, do I think that those will be the biggest pain points or the first thing being tackled? No, I suspect that the biggest DevEx problems we have right now are around the actual instrumentation API, but I don't want to presume things that aren't backed by data.

That sounds good to me, but that's not what I read from the proposal yet. Since this already has a lot of approvals I don't think that has to be changed, and at the end I will also not insist on calling this SIG differently, I just wanted (and still want) to call out that the name "Developer Experience" has a very very broad scope and SIG members need to be prepared to push back with many requests in that domain that they find out of scope, either point in time or general. To give another example that for me is within "DevEx": what if developers think that the biggest problem is that there is no developer friendly OTLP endpoint, will something like OTEP 230 be in scope?

[austinlparker]

That sounds good to me, but that's not what I read from the proposal yet. Since this already has a lot of approvals I don't think that has to be changed, and at the end I will also not insist on calling this SIG differently, I just wanted (and still want) to call out that the name "Developer Experience" has a very very broad scope and SIG members need to be prepared to push back with many requests in that domain that they find out of scope, either point in time or general. To give another example that for me is within "DevEx": what if developers think that the biggest problem is that there is no developer friendly OTLP endpoint, will something like OTEP 230 be in scope?

That wouldn't be in scope because it's quite clearly something that isn't related to the developer experience of OpenTelemetry itself. Out of scope work for the project doesn't magically become in scope because a SIG was formed.

Respectfully, this naming discussion feels like bikeshedding. The initial charter for the SIG in the proposal is clear and concise, and the name 'Developer Experience' clearly distinguishes the goals and purview of the solution space. In the event that this SIG is wildly successful, perhaps it will continue to deliver improvements on Developer Experience in other areas of the field -- there are certainly many of those. However, I continue to see no clear rationale to change the name to something that potentially restricts the solution space and scope of this project before we've even had a chance to do discovery on what problems should be tackled.

[svrnm]

Respectfully, this naming discussion feels like bikeshedding.

I understand that you see it that way and I apologize for getting that deep into this discussion.

I was asked to express my concerns publicly and to unblock this PR, so I did exactly that:

I do not have a strong concern with it, I just want to make sure that "Developer Experience" how I understand it and what the SIG plans to do is congruent.

I got my answer, so from my site there is no further need to discuss this. I approved the PR already with the initial name and so I only can say that my approval remains.

[austinlparker]

Sorry, that was less directed at you and more the generic reader.

[jpkrohling]

I'm approving this as well: I think we had enough discussions around the topic and the naming, and there's no better name for this SIG. The potential confusion with Contributor Experience can certainly be clarified quickly when it happens.

[tsloughter]

I don't see any unresolved comments but this is still blocked on 3 reviews.

[jpkrohling]

I'm merging, as I can count 5 GC approvals already and no concerns from the other 4.