Case Study

Yahoo User Profile

Site

All the verticals, e.g., Yahoo News, Finance, Sports and Lifestyle, across Yahoo networks

Canvass, the commenting system of Yahoo, is an important touchpoint on Yahoo as it increases user engagements when users interact with the comments. In addition, Canvass is implemented across Yahoo networks, e.g., Yahoo News, Sports, and Finance, which has more than 700 million monthly active users (MAU). Though Canvass has been a success of Yahoo, the design of the user profile hasn't updated for a long time. Through the user research, we learned that users don't trust the comments on our platform, and one way to address that is to show the credibility of the users. Therefore, the team is looking for a way to improve the design of a user profile while showing the credibility of the user.

My role

As a Senior Product Designer, I was in charge of the end-to-end designs. My responsibilities include:

 

PROJECT SCOPING

I helped the Product Manager define the project goal, problem statement, acceptance criteria and deliverables. 

 

WIREFRAME, PROTOTYPING AND TESTING

I created all the wireframes based on user research. To ensure the design reached the project goal, I created interactive prototypes and conducted several rounds of testings.

 

INTERACTION AND VISUAL DESIGN

I had several visual design explorations and iterated based on other designers' feedback. I illustrated detail interaction designs via Principal, an animation software.

 

COLLABORATION

I partnered with developers and made sure the design is implemented accurately.

 

The old design of a user profile. There was not much information about the user.

Challenge

Before jumping into solutions, the team and I must understand user pain points, feelings, and the moments of delights. I created a user journey map that helped visualize users emotions and create a shared understanding of user needs among the teams.

As we can see in this map, the reader experience stopped at the user profile. It's because there's little information about the user. As a result, readers can't tell the credibility of the person.

Once we successfully design the user profile that allows users to tell the credibility of users. What's next? Do users want to stay in touch with other users?

Project phase

The team and I decided to break the projects into two phases:

  1. Phase one is to redesign the user profile so it shows the credibility of a user.

  2. Once we show the credibility of a user, phase two is to test if users want to get updates from the people they're interested in.

Phase 1

Wireframes

In phase one, the user research team identified the elements that could show the credibility of a user. I then created different options for displaying these elements.

How to present the reference article?

In the user activity, we show the user's activity and a reference article. I had three options to present the information.

Option 1: Show more emphasis on the article with an image thumbnail

Option 2: Show only the article title

Option 3: Show the article title with an icon indicating the type of media

Product decision

We as a team discussed these three options and felt that option 3 would give more attention to user's activity rather the article. As a team, we made the product decision that at this point, we only want to show the credibility of a user. Therefore, option 3 is the best to reach this goal. If we want more article views in the future, we can revisit this design.

Design challenge on the content order

One of the biggest design challenges we encountered was to find the content order that makes sense for the users. There were three types of actions in the user history: user's comment on an article, user's upvote on someone's comment, and user's reply on someone's comment. Should we put user's action always on top? Or should we put content in the order that makes sense to the readers? I was able to find examples for each option,  but what do users think in our case? Moreover, does the layout reach the project goal, "showing the credibility of a user"?

Testing, testing, testing

I set up a remote user testing session with 10 participants. There were three objectives of the test:

  1. Could users tell the credibility of the user by looking at the profile?

  2. Does the content order make sense to users?

  3. Are there any usability issues

I intentionally put in real comments from real users and comments that can be confusing in order to discover the best content order in the worst scenario. Here's an interactive prototype example (click on John Doe on the 2nd comment).

Learnings from user testings

Through user testings, we learned:

  1. When there's a badge next to a user, our research participants find the user more credible. Most participants didn't need a label for the badge to understand the meaning of it.

  2. When the research participants are interested in the commenter, they want to see the profile of the commenter. They don't want to follow the user immediately.

  3. Research participants felt the metric gave a good summary of the user. At the same time, they want to see the number of downvotes.

  4. Participants knew how to mute or block a user.

  5. For the content order on user's comment, user's reply on someone's comment and user's upvote, participants' preference are: reference article on top > reference article at the bottom > no reference article.

    Reference article on top was the most intuitive. When the reference article was at the bottom, it always took the participants a while to comprehend the information. When there was no reference article, it was just confusing to the users.

  6. Most participants had screen resolution around 1280 wide. Therefore, when the user profile was in a modal, it took most of the screen and was not an issue to them. Furthermore, they understood the flow of opening and closing the user profile.

  7. All participants had the correct understanding of follow.

  8. The visual design is clean.

Visual design explorations
Fine-tune interactions
Final design

Phase 2

Fake door test

Instead of spending months on building the follow feature, we need to find out first to see if users want to get updates from the people they're interested in. To test the idea, I designed a "fake door" test. The team tracked the click-through-rate of the button and measure the potential impact of the following feature.

In the above example, I added a fake "Follow" button in John Doe's hover window. 

In the above example, I added a fake "Follow" button in John Doe's profile.

Results from fake door tests

We diverted 1% of traffic to this experiment. Out of all the users who were in the experiment, 1.5% of users opened the hover window, and less than 1% of users got to the user profile window. For the users who opened the hover window, we got 10.3% of CTR. For the users who opened the profile window, we got 14.2% of the CTR.

With this CTR, the team can evaluate if the time and cost of development would justify the potential impact on the following feature.

Based on the result, we got the signal that users do want to connect or get updates from other users. However, the following feature may not be the only mechanism to reach this goal. We should continue to explore and test other mechanisms like pushed email notification, passive alert or direct message.

Copyright © Julia Wang, 2020