Understanding What Features to Improve for an Internal Cloud Platform Product

My Recommendations Resulted in Increased Software Engineer Productivity

  • A time savings of 20 minutes per session was realized

  • An average of 500 sessions a day were conducted before improvements and increased to an average of 1500 sessions a day after improvements were deployed

  • Based on my recommendations 4 features to improve were either confirmed or added to the product road map

I Learned From Users What Features to Improve

  • Users expected conventional functionality

  • The product was comparable to a commonly used product by the general population

A Poorly Designed Internal Cloud Product Resulted in Loss of Software Engineer Productivity

  • Product lacked expected functionality

  • Loss of productivity from Software engineers because of poor usability

    • Low user engagement with product

    • The time to deployment of products was extended

The Business Objective Was to Increase Software Engineer Productivity

  • My 3 goals were to:

  1. Learn what features to improve and add to decrease time on task per session with the cloud product

  2. Learn what features to improve and add to increase engagement with the cloud product rather than external sources that could lead to public sharing of proprietary company information

  3. Confirm and add product features to the product road map

Team Members

  • Product Owner 

  • Lead Software Engineer 

  • Software Engineer

Methodologies

  • User Interviewing

  • Usability Testing

  • Contextual Inquiry

Study Duration

5 weeks

Study Presented

01/06/2023

Sample roadmap- does not reflect product in this case study

I Worked with the Product Owner to Identify 3 User Segments to Recruit Study Participants

  1. High Volume Users: 90+ uses

  2. Intermediate Users: 6-89 uses

  3. Low Volume Users: 1-5 uses

My Moderator Guide Addressed the 3 Business Goals

  • I wanted to understand:

  1. The mental model users had for the internal Optum product

  2. How the product could be improved to satisfy that mental model

  • Sessions included interview questions, usability testing, and some time for contextual inquiry

    • I planned to ask impromptu follow up questions as appropriate

  • Interview questions:

    • I asked 3 questions to gather background information about their role and their use of the product

  • Usability testing:

    • I asked participants to complete 5 tasks using the live product and provide feedback about functionality

      • I collected data for both their verbal feedback and their on-screen interaction with the product

  • Contextual inquiry:

    • I asked participants to demonstrate for me how they used the product in the recent past and provide feedback on what worked and did not work for them

      • I asked follow up questions to gain further insight into observed behavior and verbal feedback

My Findings Addressed Study Business Goals

1. Learn what features to improve and add to decrease time on task per session with the cloud product

I recommended that the Optum product look and function more conventionally like a comparable well-known product used by the general population.

2. Learn what features to improve and add to increase engagement with the cloud product rather than external sources that could lead to public sharing of proprietary company information

I recommended 13 new or improvement to features that led to increased engagement with the Optum product.

3. Confirm and add product features to the product road map

I recommended 13 new or improvement to features that were added to the product road map for the coming year.

  • The product did not function as expected leading to increased frustration and additional time using the product per session

  • Participants compared the Optum product with a well-known product used by the general population, citing unconventional functionality of the Optum product

    • This was feedback that was unprompted during sessions from every participant

  • The product did not allow Software Engineers to access commonly used resources, resulting in less engagement

  • The product did not include conventional features that helped Software Engineers meet their needs, resulting in less engagement

  • Use of the product often led to dead-ends, prompting use of publicly used products that exposed proprietary company information

  • I worked with the PO of the product to add new and improvement to features to the road map

  • I strategized next steps, including further studies to test the efficacy and engagement to all changes made to the product

I Really Enjoyed Working With the PO of the Product For My First Optum Study

  • The PO provided all information and resources I needed to be successful

  • They were open to future studies to continue improving the product (unfortunately I was laid off before I could run further studies)

Building My Study Plan Was Better Using a Word Processing Document

  • In my previous UXR role I had been trained to use Confluence to build my study plan

  • At Optum I was given a study plan template in a Word document

  • I initially created my study plan in Confluence but found that using a word processing doc provided a better experience

    • Sharing it was easier as team members didn’t need to create an account to access a Confluence link

    • Readability was improved compared to the non-linear layout of the Confluence study plan I had been taught to build