BetterUp Conversational Analytics

Background & Objective: BetterUp is a human transformation platform providing coaching and personalized learning. Conduct design experiments to validate and find a use case for conversational analytics technology.

Impact: Validated use case for conversational analytics technology and 1 design partner signed up for future launches.

Time: 3 months

Skills: User Research, UI design, surveys, A/B testing

1. Discover

To start, we gathered internal stakeholders from various teams for a large brainstorm, with a few key questions: How might we use this technology to benefit three stakeholders; our members receiving coaching, our coaches, and our partners sponsoring coaching. Another key question around this project was: how might we ease privacy concerns over video recording?

2. Ideate

Problem Definition:

What value prop could we productize for this conversational analytics technology to benefit our stakeholders?  

From the brainstorm, we synthesized to a few key ideas and created storyboards to illustrate potential directions to take this technology.

We stepped up the fidelity by creating some marketing landing pages for 3 main concepts, with 3 detailed versions for each. For each, we created 3 versions and tested this as marketing landing pages on UsabilityHub. The 3 concepts were:

1. Session notes made easy (transcript notation)

2. Practicing communication skills (insights on talk metrics)

3. Learning more about yourself (insights around content of sessions)

Testing Marketing Landing Pages on UsabilityHub

Initial 3 concepts:

We learned these users have similar concerns as our members around data privacy and sensitive topics/sentiments. They thought of transcript as an added feature, not a standalone product. Finally, concrete, tangible applications of CA (ex: communication) were easier to understand for testers and therefore tested more favorable.

3. Prototype

We learned that the value of transcript could be folded into the other 2 concepts. Our next task was to create mockups of how the 2 winning concepts would live in the BetterUp app.

Testing prototypes with BetterUp members

Concept 1: Session Insights

Concept 2: Skills Practice

4. A/B Testing

Based on our learnings from testing landing pages on UsabilityHub and BetterUp mockups on members, it became clear in both instances that the use case of leveraging this technology for tactical skills practice of communication provided more tangible value, and importantly, shifting the focus away from the content of the conversation, which opens to more issue of privacy / sensitivity. We made a Braze campaign to test the opt in rate with the full suite of proposed features (coach + tech) vs a control group with only tech features. In the end, we saw the version inclusive of the coach showed higher opt in rate with:

• 96% total opens, 44% unique opens

• 9% total clicks, 6% unique clicks

5. Impact

We ended up signing up partner that was willing to co-design a pilot with us. This work also paved the way for a squad to pick up from where we left off and also inspired a Hackathon project!