Introduction to Mobile Analytics & Reporting
Introduction to Mobile Analytics & Reporting
I recently spoke as a panelist at the MG West conference on mobile game analytics and after the panel was very surprised by how few companies and people in the industry seemed to know about analytics overall as well as how to properly execute an effective analytics program.
In this post I'd like to answer the first two of the following key analytics questions:
How should you think about analytics at a high level?
What should you be measuring?
What metrics do you need for a successful game?
In a future post, I'll also address the third question from the above.
1. How should you think about analytics at a high level?
First of all, you should take the approach of trying to think about analytics from a Role and Application perspective. In other words, who is looking at the data (Role) and what are you using the analytics for (Application).
Let's talk about some of the key applications for a moment first:
Audit: How well is the game doing and how should we think about the game?
User Acquisition (UA): What are the target metrics we're shooting for and how and how much should we spend to acquire users?
Optimization: How do I uncover insights about how to make the game better and how do we do that?
Anti-Hacking: How can I detect cheaters or hackers in the game?
In a typical mobile game company setting this may break down as follows (simplified view below):
In an ideal scenario every Application/Role would have a separate dashboard view that helps focus the user on exactly what they need to know about the game being analyzed.
2. What should you be measuring?
Let's now dive into a bit more detail about the typical metrics you should look for in each application area:
Audit:
User:
DAU (Daily Active Users): How many users used the app on a particular day
DAU/MAU: DAU / Monthly Active Users; to get a sense for marketing and user loss
Monetization
ARPDAU (Average Revenue Per Daily Active User): Daily Gross Revenue / DAU
ARPPU (Average Revenue Per Paying User)
LTV or some shorter term LTV proxy e.g., 30 Day Cohort ARPU
First Day Conversion
Retention:
D1/7/30: Retention on Day 1, Day 7, and Day 30. This is computed by calculating the actual % of users that logged into the app on the specified day from day 0
% Loss: % of users that are considered inactive after not logging in after a specified # of days e.g., 7 days
Engagement:
Sessions per Day: # of sessions / DAU
Session length: Average amount of time spent by each user in a session
Session length by level
Stability:
Crashes per day:
Server: Number of server or server process crashes per day
Client: Number of client crashes per day
Crashes per 1K (or other # depending on your traffic) DAU
Top 3 crash types: You should try to characterize crashes and isolate where crashes are occurring.
Client: It's typically more difficult to characterize a client crash but at the least you should send up a client dump to help developers try to figure out what happened.
Server: Server crashes are relatively easy to isolate and you should be able to tell even by the line number where problems occurred
Performance:
Load time for X by device type: One of the biggest sources of user loss is wait times for loading. After playing your game you should try to identify a few potential problem spots and opportunistically track load/wait times for those areas.
Device type: By characterizing load/wait times by device you can see if there are measurable differences in KPIs based on different wait times you will see on the various device types
UA:
From a UA perspective, we like to measure source effectiveness and do source/user attribution. Basically how effective is a $ I spend to acquire users? What is it getting me?
Marketing:
CPI: Cost per install
Effective CPI: CPI including organics, social, etc. that may reduce the overall cost for a user install
LTV (or LTV proxy)
Attribution
UDID/IDFA lists: You should always check to see if you actually got the users that your vendor claimed to have sent you
Is LTV - Effective CPI > 0?
3 Side Points:
For the majority of the top grossing games on iOS App Store the answer to the above should be: NO. Unfortunately, we still have a broken App ecosystem from Apple... but I digress
Many companies totally screw up LTV calculation and make forward looking projections where they project an overly optimistic LTV, overspend on UA, and in the end get screwed (*cough* *cough* GREE *cough* *cough*)
CPI in itself is not the full story as cheap CPI may come with poor traffic. Do not make the assumption that your overall LTV should be applied to any new source of cheap CPI traffic (*cough* *cough* Tapjoy bots *cough* *cough*)
3. Optimization:
This is the most custom part of building analytics for your game and will require a deep understanding of your game to determine what specific metrics are necessary. The overall objective is to tune/balanc:
Game Economy
User Progression e.g., how fast are users moving through the game in PVE, PVP, any other loops
User Activity e.g., what users are actually doing in your game... how much, how long?
Game Content/Balance
Monetization
Retention
Social/Viral
Here's a typical list of stuff I came up with for some of my games:
Game Economy:
Sinks and sources of soft currency by level
Stack bar graph of each source
Item by popularity (last 3 days, overall)
Soft Currency
Hard Currency
User Progression:
By Level:
% of players
% of plays
Average amount of money
Highest amount of money
Average net worth (including item value)
Highest net worth
Average # of days
Average # of sessions
Average # of battles
By Loop (e.g., PVE)
Show average progression by Level
E.g., Level vs. Battle # (PVE progression) as a % of users and as a total user count
Show highest progression by Level
Average mastery level by enemy/map
User Activity:
Sessions:
Average sessions per day
Highest sessions per day
Average time per session
Sessions by Level:
Average sessions per day by level
Average time per session by level
Activity Type
% of activity type e.g., PVP vs. PVE vs. Other Loop
Game Content/Balance:
% character type owned
% win by character type
Show top 10 squad types (e.g., {archer, mage, knight}, etc.) by win %
Show top 5 character types by win %
Show bottom 5 character types by loss %
Monetization:
ARPDAU
ARPPDAU (Average revenue per paying DAU)
% of users who purchase
% of users who purchase by level
% of users who purchase by source
ARPU
1 day, 7 day, 30 day
LTV
Average $ amount spent
$ spent by level
First day buyer conversion %
Store
% first item purchased
% spend by category
Top 10 items purchased
Bottom 5 items purchased
Store heat map by clicks (where do users click the most on the store)
Retention:
Tutorial completion
1 day, 7 day, 14 day, 30 day
Funnel by tutorial and level
Social/Viral:
% FB connect
Average # of FB friends installed
by FB Connect users
Average # of FB posts (per user, per active user)
by FB Connect users
% of users who have FB liked
by FB Connect users
4. Anti-Hacking
All games can be hacked. Especially mobile games with primarily client-based architectures on Android that will be especially prone to hacks and cheats.
Here is a simple list of things you should track:
Revenue velocity change
If revenue by day decreases or increases by > X% (set X = 20%)
If user net worth > 10% in 1 day after level 5 alert and log
If # of sessions > 10 per day alert and log
If user increases > 1 level per day after level 10 then alert and log
That's it for now... I'll also answer the 3rd key question around typical heuristics to indicate a successful game in a future post.
UPDATED: Crash/performance analytics added to this post on 6/11/'13 based on feedback from Lukasz Twardowski. Also Lukasz makes a very relevant point that the term "analytics" is often used in our industry (and me too) where "reporting" is actually the more appropriate term. Thanks Lukasz!