Introduction
BrowserStack AppPerformance: Revolutionizing Mobile App Testing
BrowserStack AppPerformance is a cutting-edge performance testing suite for Android & iOS devices. This project aimed to provide developers and QA professionals with a powerful tool to inspect performance metrics via interactive & comprehensive reports. This case study details the product journey, from initial exploration to final implementation.
Role
I served as the sole designer responsible for the product.
The Team
Sole designer. With guidance from 1 Design Manager, 1 Product Manager, and a team of 5 Engineers, Frontend & Backend
Timeline
- Alpha: Winter 2023
- Beta: Spring 2023
- New features rolled out until Summer 2024
Mobile App Performance Overview
In today's hyper-connected world, there are multiple devices with varying different hardware and software capabilities.
Impact of poor app performance
Slow load times, crashes, and unresponsive interfaces can frustrate users, leading to a decline in satisfaction and loyalty.
Performance issues result in increased churn, lower conversion rates, and a direct loss of revenue.
Poor app performance tarnishes the businesses' reputation, reducing trust and making users less likely to recommend the brand.
What existed as a solution
Key Workflows
Initiating a Performance Test: Users start by selecting a specific build of their application. Our tool provides an intuitive interface to configure the test parameters, ensuring consistency across multiple test runs.
Standardizing Test Flows: Users can standardize their testing flows, allowing them to replicate the exact scenarios for each build. This feature is crucial for identifying performance issues and ensuring reliability.
Analyzing Historical Test Reports: Post-testing, users can access detailed historical reports. Our tool offers a comprehensive view of performance trends over time, enabling users to track improvements and identify recurring issues.
JTBD Analysis
To deeply understand the needs of our users, we conducted a Jobs to Be Done (JTBD) analysis. This framework helped us uncover the core tasks and desired outcomes that developers seek when measuring app performance.
First Product Vision
Alpha Version
The first alpha version, which was provided as a desktop application that allowed users to test apps locally, using their own devices.
Splash Screen
The splash screen that first appears when the application is launched sets the tone for user engagement, providing a brief visual introduction before transitioning to the main interface.
Splash Screen
The splash screen that first appears when the application is launched sets the tone for user engagement, providing a brief visual introduction before transitioning to the main interface.
Introduction Page
The introduction page is designed to provide users with an overview of the application's features, its purpose, and how it can benefit them. This helps in setting user expectations and ensuring they are engaged from the start.
It also allows users to login with their BrowserStack account to save their reports to the cloud.
Introduction Page
The introduction page is designed to provide users with an overview of the application's features, its purpose, and how it can benefit them. This helps in setting user expectations and ensuring they are engaged from the start.
It also allows users to login with their BrowserStack account to save their reports to the cloud.
Landing Page
The landing page invites the user to start a performance test. The first connected device and the first available app (alphabetically) are pre-selected to ensure that the user can easily initiate a test without needing to navigate through multiple menus.
Landing Page
The landing page invites the user to start a performance test. The first connected device and the first available app (alphabetically) are pre-selected to ensure that the user can easily initiate a test without needing to navigate through multiple menus.
Testing in Progress
While testing is in progress, real time metrics from the ongoing test are displayed. This is designed to give users an immediate sense of how their device and app performance is being monitored in real time, enhancing transparency and control over the testing process.
Testing in Progress
While testing is in progress, real time metrics from the ongoing test are displayed. This is designed to give users an immediate sense of how their device and app performance is being monitored in real time, enhancing transparency and control over the testing process.
Performance Report
The final, comprehensive report provided after the manual test session is complete. (Open the image to view the full length report).
Performance Report
The final, comprehensive report provided after the manual test session is complete. (Open the image to view the full length report).
Integration with other tools
App Live
Description
App Automate
AppLive allows testing mobile apps via automated scripts, making them faster and more efficient. It is primarily intended for more matured testing users.
Integration with BrowserStack AppLive
As a part of the Beta, App Performance was directly integrated into App Live, a BrowserStack offering that allows users to connect to real, bare metal devices (not virtual machines or emulators), to test mobile apps.
Welcome Dialog
The welcome dialog introduces App Live users to the performance testing feature.
Welcome Dialog
The welcome dialog introduces App Live users to the performance testing feature.
Main Page
The Performance Tab provides a way to start performance profiling.
Main Page
The Performance Tab provides a way to start performance profiling.
Test In-Progress
Just like the local, desktop experience, real time metrics are provided during the test execution, allowing for real-time monitoring of performance metrics such as CPU usage, memory consumption, and UI rendering performance (Frames Per Second).
Test In-Progress
Just like the local, desktop experience, real time metrics are provided during the test execution, allowing for real-time monitoring of performance metrics such as CPU usage, memory consumption, and UI rendering performance (Frames Per Second).
Results
A summary of results is provided, with a link to the final report for the session, or to the full set of reports from previous runs.
Results
A summary of results is provided, with a link to the final report for the session, or to the full set of reports from previous runs.
App Performance Manual Testing Dashboard
The App Performance Manual Testing Dashboard is where users can view, filter and analyze manual test reports from previous runs.
App Performance Manual Testing Dashboard
The App Performance Manual Testing Dashboard is where users can view, filter and analyze manual test reports from previous runs.
Web Report
The web report is the implementation of the comprehensive performance report within the browser
Web Report
The web report is the implementation of the comprehensive performance report within the browser
Implementation in Automated Testing
App Performance was also implemented on App Automate, BrowserStack's offering for users who used automated testing scripts.
One of the first enterprise offerings was an implementation of a dashboard for the users to envision performance trends over multiple builds of the application.
A comparision feature was added to better help understand performance differences across different builds of the same application.
Trends
The trends view provides a high level view of the performance trends over a timeline of multiple application builds over the course of time.
Trends
The trends view provides a high level view of the performance trends over a timeline of multiple application builds over the course of time.
Comparison
The comparison view allows comparison across multiple builds of the same application. Users can use this view to pinpoint performance anamolies, if any.
Comparison
The comparison view allows comparison across multiple builds of the same application. Users can use this view to pinpoint performance anamolies, if any.
Additional Features & Experiments
Features that didn't make into the final product but are worth noting