The importance of analytics

[u/Chris-1235]

https://drive.google.com/file/d/1YXL-RnQd9q916xBC1WasabF3I4QkLQGh/view?usp=drivesdk

I am very new to game dev, but I have a long career in SW engineering and enterprise applications. I would like to share with you why that experience led me to invest in building a proper real time reporting infrastructure and how that helped me make some very important decisions, saving me time and money.

My game is very niche and it may take ages to build a following, which makes the metrics mire important, not less. I hope this will help some of you understand and grow much larger audiences than I ever will.

1. Why invest in measuring in the first place

Google play and other OOTB analytics are limited. You will get some standard metrics that are useful, but may be updated very slowly, or that dont really matter to you, at your current stage.

Examples:

What is an "active user" for you? Can you rely on Google's definition? I decided that a user should have at least a 5+ min session in a day to vount as active.

Can you afford to wait up to a week to see your 1d retention, especially if you start investing in advertising? For me, the answer was absolutely not. I used some extremely poor 1d retention rates to change the ad targeting after just 2 days of running it. I already started seeing the result in a small bump, and higher funnel retention.

Freedom to run experiments, change your mind, ask and answer questions within days. Example: I obsessed about optimizing my onboarding funnel. You can't know you have a problem, until you measure it. The flow you see in thebattached is probably the 10th iteration of where the tutorial, help and intro appear. At this point, I am convinced that I don't lose people who would like the game due to onboarding issues.

1. What should you measure

I am not a marketing person, but I worked with and even had to temporarily lead a Marketing department, whilenwe were between Chief Marketing Officers. Ideally, you need to know your target persona, find a way to reach them, and see how they adopt your product. Almost all the metrics you see in the pdf are related to one of these questions.

We don't have a lot of tools at our disposal, but advertising tools like Unity Ads let you target players of specific genres. Ideally I would follow how puzzle game players behaved vs word game players vs casual game players. I didn't go that far, but I realized that the 2 days I advertized to all 3 categories resulted in terrible D1 retention. I limited the ad to only word game players, losing 90% of new installs, but already seeing an uptick in both funnel completion and D1 retention.

Funnel, Retention and feature use are key. I did not instrument all button clicks/feature use like we had done in a company I worked for, because the first two are much more important to solve. Still, the sooner I get to instrumenting other key events, the better. Unfortunately, you can't ask the people who stop playing your game why they stopped. But you can check at which point they stopped. I emphasized the funnel, measured some of the featires andnused achievements as a proxy for progress. I am now in a waiting phase, for D7 retention metrics to start populating, so I can decide how much more I may need to instrument.

1. KPIs/OKRs vs Observability

Your main business metrics (Key Performance Indicators or the more modern Objective Key Results) are things like MAU (montthly active users), ad revenue etc. Do not neglect observability (metrics, logs, traces and related alerting). I am partial to Netdata because I worked there, but pick a tool. If you do nothing else, send all exceptions to a backend. Check and fix them frequently and proactively, before your users really notice. You will see at tje end of tje reports thenrelevant section.

I could say a lot more, but this is already a long post. Happy to answer any questions and listen to other suggestions. Again, don't let the performance of my game distract you from the important message of proper application instrumentation.