Back to Methods page

Table of Contents

Contributors

Te Ma

Resources

No resources available for this entry.

Additional tags

No tags are assigned to this entry.

Understanding and defining outcomes and KPIs

Author: Kevin Hatchoua | Last edit: May 01, 2024

What are outcomes?


Business outcomes

Business outcomes are the specific achievements crucial for an organization's success. They often involve profitability, market growth, customer satisfaction, efficiency improvements, or other key performance indicators that reflect how successful a business is.

Example:
Increase the adoption of OpenShift among enterprises, resulting in a 20% year-over-year growth in OpenShift market share and a 15% rise in customer satisfaction scores for the OpenShift platform.


Product outcomes

Product outcomes are the measurable results a product or feature aims to achieve. They highlight the value or benefits the product brings to users and the business, such as increased user engagement, higher adoption rates, improved user satisfaction, or generating revenue.


User outcomes

User outcomes are the positive effects users experience when interacting with a product or service. They focus on fulfilling user needs, satisfaction, and the value gained, like making the product easier to use, saving time, increasing productivity, or improving the overall experience. User outcome statements should be created from user-sourced materials. either through user interviews or past user research work directly with that specific user group.

Example:

  • Minimize the likelihood of deploying an application with security vulnerabilities.
  • Minimize the time it takes to create ROSA clusters.

When to write an outcome measurement goal?

As soon as possible before implementation, you should start planning the measurement goal with stakeholders (Business Unit,  Engineers, Product Marketing, etc). Define a clear high level goal before you bring them down to specific metrics.  Always start from top to down, avoid bottom-up.


How to do an outcome measurement?

Here’s the sample measurement goal framework:

If Red Hat provides <name of service/product/feature>, then <type of user or customer> will be able to <user benefit or business value>.

We will know this is true when: ~3 key user flows that can be measured

Example 1: OpenShift performance example

Enhance the performance and usability of OpenShift by reducing the deployment time by 30% and increasing user engagement through the adoption of new features by 25% within 6 months of implementation.

We’ll know this is true when:

  • Achieve a 30% reduction in deployment time within 6 months
  • Increase user engagement by 25% via new feature adoption.
  • Establish a positive correlation, showing that new features contribute significantly to user satisfaction

Example 2: ROSA UX Improvements example

Customers coming from AWS, who are interested in Red Hat OpenShift, should be linked directly to the OpenShift content so they can quickly learn about OpenShift and access what they need to create a cluster.

We’ll know this is true when:

  • We see an increase in views of the pre-requisites page
    • Number of visitors to pre-reqs page
    • Correlation of pre-reqs views with Cluster Creation
  • Increase in completion of the “Create Cluster” wizard in 2 or less attempts
    • Number of Create Cluster start vs completion
  • Increase in downloads of the CLI
    • Number of clicks for downloading the CLI

Another example: API and Schema Designer - Understand


UX goals

UX goals are the targeted objectives aimed at enhancing how users interact with a product or service. They focus on improving the user journey, satisfaction, ease of use, accessibility, and overall enjoyment users have when engaging with the product.

Example:
Improve the accessibility of OpenShift, aiming to increase user satisfaction by ensuring a seamless and user-friendly experience for users with diverse needs and preferences.


Metrics

Metrics are specific, measurable data used to track, evaluate, and understand the performance or outcomes of a product or feature. They provide insights and indicate progress toward achieving goals. Metrics can include quantitative and qualitative data like conversion rates, engagement metrics, user satisfaction scores, or usability testing results.

Example:

  • Conversion Rates: Track the percentage of visitors signing up for Red Hat's trial programs specifically for OpenShift after visiting the product page.
  • Engagement Metrics: Monitor the average time spent by users within the OpenShift console and the frequency of user logins to the OpenShift platform.

User Satisfaction Scores: Collect feedback from customers using OpenShift through surveys or Net Promoter Scores (NPS).\Usability Testing Results: Conduct regular usability tests to measure task success rates and efficiency using OpenShift's various functionalities.


What are KPIs (Key performance indicators)

KPIs are the most critical metrics directly connected to strategic goals. They show how effectively an organization, team, or individual is achieving key objectives. KPIs are measurable, aligned with outcomes, relevant to performance, and usually set within a defined timeframe for periodic assessment and improvement strategies.

Example:

  • KPI target: Achieve a 30% increase in the number of enterprises using OpenShift compared to the previous year directly measuring the platform's product market fit. 
    • This KPI directly ties to strategic business goals, indicating market expansion and the platform's adoption among key enterprise customers.

Telemetry

Telemetry primarily focuses on the automated collection and transmission of data, usually in real-time, from remote sources for monitoring purposes. This data can include information about the status, behavior, or performance of devices, machines, or systems. 


Product analytics

Product analytics involves the systematic analysis and interpretation of collected data to extract meaningful insights, patterns, and information that can be utilized for decision-making, optimization, and strategic planning.

In most cases, telemetry data serves as a foundational dataset for conducting product analytics. By analyzing telemetry data alongside other product utilization datasets, such as:

  • User interaction logs
  • Customer feedback
  • Sales data

 We can derive comprehensive insights, enabling better decision-making and strategic planning.


methods process

Get in Touch!

Spotted a typo? Any feedback you want to share? Do you want to collaborate? Get in touch with the UXD Methods working group using one of the links below!

Drop us a message on the #uxd-hub Slack channel

Submit Feedback