Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build a combined Track and Submit flow on the FE, so we can use it to run our very first A/B test. #47598

Open
anmurali opened this issue Aug 17, 2024 · 46 comments
Assignees
Labels
Reviewing Has a PR in review Weekly KSv2

Comments

@anmurali
Copy link

anmurali commented Aug 17, 2024

Problem: The vast majority of signups are unengaged. Paid adoption from top or bottom up leads only happens when users are engaged with the product and derive value from it.

Hypothesis: The time it takes for a person to make a decision increases with the number and complexity of choices available. Presenting users with too many options can lead to decision fatigue, making the user experience overwhelming and less efficient.

Test: The Global Create menu is by far the most obvious and accessible option a new user explores to do something. Let’s make it a smaller, focused list that allows a user to pick one out of a very small set of choices. Instead of asking the user to decide between tracking and submitting an expense, let’s present a create expense that after the expense is manually entered or scanned allows the user to select what they want to do with that expense in the moment.

Metric: conversion in this test would be number of people that create an expense / number of people that see the create expense option Vs sum of the number of people that track or submit an expense / number of people that see track and submit as separate options in Global Create

Notes: We are already cleaning up the GC menu #47274 so we focus the user on a smaller set of realistic actions. Once we do that, we should be ready to test an idea @shawnborton proposed, which is to use one single combined flow that allows the user to first create the expense and in the next step decide to either save it for now or submit to someone.

Let's get started on building this and just put it on a beta for now and don't show it to anyone not on the beta. That way, when we finish cleaning up the GC and figure out how to set up the test, we can turn it on accordingly to the variant audience.
image

  • Just track this for now would basically mimic the track expense flow
  • Choosing someone to submit it to would then follow the submit flow
@shawnborton
Copy link
Contributor

Updated mock would look like this:
CleanShot 2024-08-19 at 08 29 13@2x

cc @Expensify/design

I think this would be a good place to start. Then I think separately we should really follow up and make some improvements to the tracked expense experience, to make it easier to send a tracked expense somewhere after you indicate that you want to save it.

@dubielzyk-expensify
Copy link
Contributor

That looks good to me :)

@melvin-bot melvin-bot bot added the Monthly KSv2 label Aug 20, 2024
@grgia
Copy link
Contributor

grgia commented Aug 21, 2024

I can help move this forward - will assign for now to track but need a bit more context

@grgia grgia self-assigned this Aug 21, 2024
@grgia
Copy link
Contributor

grgia commented Aug 21, 2024

@anmurali @danielrvidal
Curious how this would work. Is it % of all users? Or a subclass of users?

@anmurali
Copy link
Author

@grgia how we actually deploy the test is still actively under discussion here. Cause we gotta adopt a framework that we can use repeatably over time. I think for now we could develop this variant, and put it behind a beta?

@grgia
Copy link
Contributor

grgia commented Aug 22, 2024

Yeah that works @anmurali. Do we have a doc or something this links to so I can get more context?

Is the A test just the current FAB/Global Create menu?

@grgia
Copy link
Contributor

grgia commented Aug 23, 2024

I pushed a PR for the beta. Can I get confirmation that this is ready to be built so I can get one of our external agencies on it?

@melvin-bot melvin-bot bot added Reviewing Has a PR in review Weekly KSv2 and removed Monthly KSv2 labels Aug 23, 2024
@grgia grgia removed the Reviewing Has a PR in review label Aug 23, 2024
@anmurali
Copy link
Author

I wrote up a problem/ hypothesis/ solution here

@anmurali
Copy link
Author

We are ready to build this and put on a beta while we figure out how we will setup testing itself.

@grgia
Copy link
Contributor

grgia commented Aug 23, 2024

Reached out to callstack and @fabioh8010 has offered to take this on - See here https://expensify.slack.com/archives/C03UK30EA1Z/p1724426198287819

@grgia
Copy link
Contributor

grgia commented Aug 23, 2024

@shawnborton could you update the main description with the final mock and link the figma? Thanks!

@grgia
Copy link
Contributor

grgia commented Aug 23, 2024

@fabioh8010 this is the beta we will build under #47921

@fabioh8010
Copy link
Contributor

Hi, I'm Fábio - expert agency contributor - and I would like to work on this issue.

@fabioh8010
Copy link
Contributor

fabioh8010 commented Aug 26, 2024

@grgia @anmurali Do you mind sharing the content of these threads (here, here)? I don't have access to them. Thanks!

EDIT: Do you also mind sharing and giving access to the Figma link? Thanks!

@anmurali
Copy link
Author

https://www.figma.com/design/ReBRDcqMXgtvPgWBBCmBxs/Wave-9%3A-Track-Expense?node-id=512-11640&t=0d1PZheynnl6iKNr-1

I updated the issue description with what's in the threads. No need to read them! :)

@danielrvidal
Copy link
Contributor

Rather than saying Just track this for now I think we could/should use Personal tracking or Track for myself. My reasoning is 1) it's shorter and 2) I think it's more clear because just track this for now could still be submitted to someone else.

@anmurali @shawnborton @dubielzyk-expensify @jamesdeanexpensify not sure if you agree or like what it currently is more.

image

@shawnborton
Copy link
Contributor

Personally I think we need some kind of verbiage that makes it clear that you are not trying to submit it, so saying "Just track it for now" reads as though you are acknowledging that you are on a submit screen, but you don't want to submit it. Actually my original suggestion was "Don't send this, just track for now" to be extra clear here but I think someone suggested we shorten that.

I don't feel too strongly though, happy to go with whatever you all think is best.

@jamesdeanexpensify
Copy link
Contributor

'Just track, don't send (for now)' we could even remove the '(for now)' but I like keeping the affirmative in front.

@jamesdeanexpensify
Copy link
Contributor

jamesdeanexpensify commented Aug 26, 2024

I would also be fine doing 'Just track it for now' though. We could perhaps make it a bit clearer as another option by saying 'Or just track it for now'

@danielrvidal
Copy link
Contributor

Sounds like people like the affirmative so far so happy to stick with what we have right now. It just caught my eye.

@shawnborton
Copy link
Contributor

Nice, I still think I like adding the don't in there somewhere to be extra clear, so I think something like Just track it, don't send would be my vote.

@jamesdeanexpensify
Copy link
Contributor

I see the Figma file and I asked for edit access. We could look at a few options to see what looks nice?

@jamesdeanexpensify
Copy link
Contributor

image

@dubielzyk-expensify
Copy link
Contributor

Agree with that Mr Borton

@quinthar
Copy link
Contributor

Can we make the title "Submit expense" so we follow the overall language of the product, and not introduce some new concept of "sending an expense to someone"?

@shawnborton
Copy link
Contributor

Sure, that would look like this:
CleanShot 2024-08-27 at 08 19 44@2x

@grgia
Copy link
Contributor

grgia commented Aug 27, 2024

@shawnborton @jamesdeanexpensify
Could you ping me when we've confirmed the final mock up so I can keep this moving? Thank you!

@jamesdeanexpensify
Copy link
Contributor

I would be fine with this!

@shawnborton
Copy link
Contributor

Yup, same!

@fabioh8010
Copy link
Contributor

Update: Working on the proposal / action plan to refactor and develop the combined flow. I plan to post it here tomorrow.

@fabioh8010
Copy link
Contributor

Update: Validating / polishing the final proposal after encountering some problems.

@fabioh8010
Copy link
Contributor

fabioh8010 commented Sep 3, 2024

Edited by proposal-police: This proposal was edited at 2024-09-03 21:57:42 UTC.

Proposal

Please re-state the problem that we are trying to solve in this issue.

The vast majority of signups are unengaged. Paid adoption from top or bottom up leads only happens when users are engaged with the product and derive value from it. The Global Create menu is by far the most obvious and accessible option a new user explores to do something.

What is the root cause of that problem?

The time it takes for a person to make a decision increases with the number and complexity of choices available. Presenting users with too many options like Track expense and Submit expense options in the Global Crea menu can lead to decision fatigue, making the user experience overwhelming and less efficient.

What changes do you think we should make in order to solve the problem?

1) Change FloatingActionButtonAndPopover to pass different data to PopoverMenu's menuItems prop if the beta is enabled. If the beta is enabled we should render the Create expense option instead of Track expense and Submit expense.

2) Create a new create IOU Type in CONST.ts. This new type will represent the new global Create expense flow.

3) Modify IOUUtils.isValidMoneyRequestType() to account for the new type (maybe we can also check if user has beta here), so it will allow AccessOrNotFoundWrapper to let the user access the new flow when accessing the IOURequestStartPage page. Additionaly, a new header title will be added for IOURequestStartPage.

4) For each tab (Manual IOURequestStepAmount, Scan IOURequestStepScan, Distance IOURequestStepDistance), modify the code to navigate the user to IOURequestStepParticipants page if the IOU Type is create.

5) In IOURequestStepParticipants page, modify MoneyRequestParticipantsSelector component to add the Just track it button (using MenuItem) to its SelectorList's headerContent prop if the IOU Type is create.

This MenuItem, when clicked, will call the new onTrackExpensePress prop that will be used in IOURequestStepParticipants to navigate the user to IOURequestStepConfirmation page passing the track IOU Type, thus allowing the user to finish the flow as they was tracking an expense.

In case the user select a participant, they will be navigated to IOURequestStepConfirmation passing the submit IOU Type instead, so the user can finish the flow as they was submitting an expense.


With this proposal I believe we can achieve this goal with minimal changes in the code while having support for both Create expense and Track expense / Submit expense options.

There are other minor changes that will be required but the main ones are these. I believe all this can be done in one PR because there isn't too much work / refactoring to do as I noted in my investigations and tests.


@grgia I have one question though: When we click the Track expense button from Global Create, we are firstly presented with a Track onboarding modal that includes a video.

1) Should we show them when the user navigates to the confirmation page (after clicking the Just track it button)?

2) If yes, maybe we would need to have a new video for this modal as the current one won't make sense?

As discussed in this issue, the Track Expense onboarding modal won't be shown in the combined Track / Submit flow.

What alternative solutions did you explore? (Optional)

N/A

Copy link
Contributor

github-actions bot commented Sep 3, 2024

{grgia} Your proposal will be dismissed because you did not follow the proposal template.

@shawnborton
Copy link
Contributor

@fabioh8010 I think for this test, it would make sense to just remove the video from the combined flow entirely? Curious what @anmurali thinks too.

@fabioh8010
Copy link
Contributor

Yeah I don't think we need the video modal there, it looks redundant to me.

@anmurali
Copy link
Author

anmurali commented Sep 4, 2024

Yes, let's remove that for sure.

@melvin-bot melvin-bot bot added the Overdue label Sep 5, 2024
@fabioh8010
Copy link
Contributor

@grgia bump 😄

@grgia
Copy link
Contributor

grgia commented Sep 9, 2024

@fabioh8010 Proposal looks good, thank you! Would you update it to specify that we will remove the video from that flow?

@grgia
Copy link
Contributor

grgia commented Sep 9, 2024

Progress Update

  • Beta is ready to go
  • I chatted with @fabioh8010, and implementation will start tomorrow.
  • I created an issue here for the feature implementation

We should have the feature ready to go soon for when we decide how we want to A/B test

cc @anmurali

@fabioh8010
Copy link
Contributor

@fabioh8010 Proposal looks good, thank you! Would you update it to specify that we will remove the video from that flow?

Updated!

@anmurali
Copy link
Author

@fabioh8010 - responding to your question here

image image image

@dubielzyk-expensify
Copy link
Contributor

Sorry if I'm coming in late here, but just curious what are we actually A/B testing here? Is it changing the wording from Submit expense to Create expense?

@anmurali
Copy link
Author

Submit/ Track as separate options vs. Create. And where there is only one or the other of the former - that vs. Create.

@grgia
Copy link
Contributor

grgia commented Oct 1, 2024

@anmurali we've merged the feature. Once it's deployed and passes QA I can merge the code releasing it to half of users

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Reviewing Has a PR in review Weekly KSv2
Projects
Development

No branches or pull requests

8 participants