FinTech is very competitive and dynamic environment. Developing a mobile payment application is like a never ending fight. A picture that pops to my mind at the beginning is two guys punching each other but to be honest, there are more opponents here and you don’t know where the hit comes from.
So the question arises how can you ensure that your product gets the best User Experience? What makes people use it?
At this point I would like to paraphrase Michał Sadowski, a very successful polish entrepreneur who made it to the top with his Brand Monitoring Tool – Brand24, who said more or less something like this:
Don’t try to invent something new. Build a product that does what other products do… but better
So do it a little faster, smoother, more convenient, more user-friendly. Don’t you think of sports already?
Our dreams are small.
One millisecond, one quarter-inch, one breath.
We’ll spend a lifetime chasing them.
Because these small things,
Make all the difference.
What kind of sport is the Product Design? What is to be measured? Conversion funnels? Time on tasks? Error rates? How should you approach such a complex subject? First of all, there isn’t one good metric (maybe except revenue ;) ) that can be applied to the product. Secondly, the applied metrics will vary across the products but you can apply the universal approach that will ensure you are gathering a good analytical overview on your app or a website.
The EARTH
Let’s illustrate the UX metrics approach based on a case. Imagine an app having such features:
- In-store payments
- Parking payments
- E-commerce platform
Now, let’s think what are the possible flows here:
- On-boarding
- Payment itself
- Transaction history
- Classical e-commerce flows
- content discovery
- checkout
- delivery
- complaints and returns
- etc.
We can apply the certain metrics to the certain features of the app separately but they will be distorted by other functionalities, so we need to think globally. Let’s introduce the HEART metrics which consists of 5 key areas to investigate that will ensure the proper Product Experience:
- Happiness
- Engagement
- Adoption
- Retention
- Task success
I would like to start from Engagement, so in our case it will be EARTH approach:
- Engagement
- Adoption
- Retention
- Task success
- Happiness
Engagement
Typically Engagement is connected to the average session time. So it tells how much time a user spends interacting with the app. But does it really tell the story? This metric alone can give false outcomes.
I’ll tell you about a real-life Hot-Dog Paranoia
Source: Flickr
There is always a situation when I am approaching the cash register at the gas station that the attendant tries to sell me something more than just a gasoline. I understand they have certain conversation scripts and they need to follow them. It usually goes like this:
A – Attendant
L – Me
A: Would you like a hot-dog as well?
L: Yes, please
A: American style or French style?
L: American, please
A: with a regular sausage or a dry smoked sausage?
L: A regular one, please
A: Would you like a ketchup, a sauce or a mayonnaise with it?
L: A sauce
A: A tartar sauce, a mild sauce or a garlic sauce?
L: Thank you, I’ll pass. Just a gasoline, please
I was very engaged for a period of time but there was no conversion and I became annoyed as you may presume.
So the average session time must be aligned with the task success metrics to give you the fuller picture on the transaction. Where does it lead? How about observing the average payment frequency? I think it gives a completely different angle on the subject. When I combine this metric with the session time I get the idea if the users are engaged and if this engagement leads to successful payment.
But that’s not all. The users are engaged if they explore the app features. Therefore there is another metric to be combined here. Average time spent in non-core features like browsing through e-commerce platform, updating profile information etc.
So to conclude we will have here:
- average session time
- average payment frequency
- average time spent outside the payment flow
Adoption
It seems pretty obvious. We should measure amount of onboarded users vs all users who downloaded the app. Well not quite. This is one angle but what about current users? There are always changes being made in the design, front-end, back-end etc. In order to have knowledge how people react to those changes it’s good to get the idea on users who updated the app vs those stay with the current version. In the world of hybrid apps not every change requires the new app release in the proper app store. So if there is a new feature or a redesigned one we should measure how many users started to use a feature with a success vs all of the users having access to the feature.
To measure adoption we need to analyze:
- onboarded users vs all users downloaded the app
- number of current version app in use vs all versions in use
- users tried new feature with a task success vs all users having access to the feature
But those metrics will not tell us if people want the certain service for the long-term. We need to analyse retention as well.
Retention
Usually retention is defined by the following formula:
Retention Rate = ((CE-CN)/CS)) X 100
CE = number of customers at end of period
CN = number of new customers acquired during period
CS = number of customers at start of period
How to apply it smart to your product? Of course you should have a global overview but applying such a metric for every feature will give you a good basis for business decisions on answering the following questions:
- Should we develop this feature?
- Should we improve this feature?
- Should we do the A/B testing on this feature?
Task Success
Now the fun begins. In order to analyse task success it is needed to apply the following metrics:
- Time on task (How much does it take to get through the process? How much time does it take finish a single step?)
- Error rate (What is the average amount of errors user make during a process? Where users tend to make errors?)
- Back Button tap rate (Is the process seamless? Do people come back and forth? Why? Is the process clear?)
- Users completed a process / All the users entered the process (What is the efficency of this process? How many people abandoned it?)
Happiness
Are you really able to measure happiness? Well, indirectly yes of course but it will require some manual work. You’ll need investigate and translate into the numbers following signals:
- Customer Service course of dialogues
- App stores reviews
- Social Media users reactions to your brand/app/feature
- Brand/app/feature discussion over the web
There are of course the simple quantitative metrics such as:
- Net Promoter Score
- App store star ratings
but analysing what people saying about your app and in what context will give you much more insights than simple numbers.
Summary
There is a lot to do to have a clear overview on your product but without that work done it’s impossible to make wise decisions for the future. Moreover there is no good dialogue with end consumers without their behaviour analysis. It’s good to have tools, it’s good to have graphs but nothing will replace observing users in their natural environment when they use the app. So don’t rely on the data only. Get out of your cave and reach out to your end consumers.
And don’t do it alone. UX is a team sport.
Icons downloded from flaticon.com
