How to write a tracking plan for a new feature

Maya

Payment reminders ship in three weeks: Halves nudges whoever owes money, and they can open it and settle up. After launch I need two answers. Do reminders get people to settle faster? And do they annoy anyone?

You

Then let's list the events: reminder sent, reminder opened, settle-up. Anything else?

Alex Tutor

Go one step back. A tracking plan is built from the questions down: first the questions, then the events and properties that answer them, then where each one is logged, then the metric definitions, and last a check before launch. An event nobody needs for a question is just noise.

Maya's two questions split into four. How many reminders reach people and get opened? How many reminded debts get settled, and how fast? Is that faster than without a reminder? And how many people mute reminders?

Maya

The last one worries me most. If a reminder spoils the mood in a flat, I want to know in week one, not in month three.

Alex Tutor

Question 1

Put the steps of building a tracking plan for payment reminders in order.

Tap an item, then tap where it should go. You can also drag or use the arrows.

Alex Tutor

Now the events. The server sends a reminder, so reminder_sent is a server-side event: it's logged even when the push never arrives. As with recurring bills, it isn't user activity. reminder_opened is logged by the app when the person taps it.

For "how fast" you need a link. Give every reminder a reminder_id, and put it on reminder_opened and on settle_up_recorded when the settle-up starts from the reminder. Without that link, reminded settle-ups look like any other.

Theo

And annoyance? Nobody taps a button called "I'm annoyed".

Alex Tutor

No, but they act on it. Log reminders_muted when someone turns reminders off, with scope: this group or all groups. Mutes, and people who stop opening, are what annoyance looks like in data.

Alex Tutor

Question 2

Match each question about payment reminders with the event that answers it.

Pick an answer for each row. Choosing an answer that is already used moves it to this row.

How many reminders did Halves try to deliver?
Choose an answer
How many reminders did people tap?
Choose an answer
Which debts were settled after a reminder?
Choose an answer
How many people turned reminders off?
Choose an answer

You

Where does "reminders are off" go? On the event or on the user?

Alex Tutor

Both. The mute is an event: it happened at a moment. Whether reminders are on right now describes the person, so it's also a user property, reminders_enabled. The debt amount and whether it's the first or second reminder describe one reminder: event properties on reminder_sent.

One rule: put on each event everything you'll later want to filter or group by. You can't add a property to last month's events.

Maya

Then I'll ask this on day one: do second reminders get muted more often than first ones?

Alex Tutor

Question 3

Maya asks: "Do second reminders about the same debt get muted more often than first ones?" The draft plan has reminder_sent (reminder_id, recipient_user_id, group_id) and reminders_muted (scope). What must be added to answer her? Select all that apply.

Choose all that applyapp_version on reminder_sentreminder_number on reminder_sent (1 for the first reminder about a debt, 2 for the second)amount_usd on settle_up_recordedreminder_id on reminders_muted: the last reminder the person got before muting

Alex Tutor

Next, write each metric as a definition, like last lesson. Mute rate: of the unique users who got a reminder this week, the share who muted within 7 days. People, not mute events: someone toggling five times is one person.

You

And "faster than without a reminder"? We could compare reminded debts with the others.

Alex Tutor

Careful: reminders go to debts that are already late, so the groups differ from the start. A fair comparison needs a holdout group: a random share of debts that would get a reminder but don't. Plan it now, with a reminder_held_back event.

Maya

Fine, a small holdout. Now write me the one I care about: time to settle.

Alex Tutor

Question 4Short answer · AI-checked task

Write the metric definition for time to settle after a reminder in 2–4 sentences. Say which events it starts and ends with, who is excluded, what happens to reminders that are never settled, and which number is reported.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Alex Tutor

Last step: QA. On a test account, trigger each event and look at the raw rows. Does it fire once, not twice on a retry? Are the properties filled, with no empty reminder_id?

Two more. Test accounts must be flagged as internal traffic, or QA taps become "opens". And times are logged in UTC, so keep the user's timezone to see when a reminder arrived locally.

Theo

That one matters. If reminders land at 3 a.m. in Bangkok, mutes jump and we'll blame the wording.

You

So QA checks the data, not just whether the feature works.

Alex Tutor

Question 5

A developer's draft plan for reminders has one event, reminder_shown, logged by the app when the reminder appears on the phone. The open rate is computed as reminder_opened / reminder_shown. What is the main flaw?

Choose one answerThe event name should be in the past tenseReminders that the server sent but that never reached the phone are invisible, so the open rate looks higher than it isThe app can't log anything while it's closed, so reminder_opened will always be zeroOpen rate should be computed per event, not per user

Maya

Next quarter we add trip budgets. I'll want the same kind of plan for it, before anyone writes code.

Alex Tutor

Then run the same five steps: questions, events and properties with their source, one written definition, and the checks before launch.

You

Starting from Maya's questions, not from the screens.

Alex Tutor

Exactly. Screens change; the questions are what the data is for.

Alex Tutor

Question 6Case · AI-checked task

Write a short tracking plan for Halves trip budgets.

Halves is adding trip budgets. In a group with group_type = trip, any member can set a budget in USD for the trip and change it later. Everyone in the group sees how much of the budget is left as expenses are added. When 80% of the budget is spent, the server sends a push alert to all members.

Maya will decide after two months whether to keep the feature, put it into Halves Plus or remove it. She wants to know:

Do trip groups actually set budgets?
Do groups with a budget keep adding expenses during the trip, rather than giving up halfway?
Is the 80% alert useful, or do people mute it?

Write the plan (200–350 words):

The questions the data must answer (you can refine Maya's).
The events and properties for each question, with where each event is logged (app or server). Mark any user properties.
One metric definition written in full: events, who counts, window, exclusions.
At least two QA checks before launch.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Alex Tutor

That's the whole module: with events you trust, written definitions and a plan before launch, every number after it has a chance of being right.

Alex Tutor

That’s the lesson. You answered every task — nicely done.

Next lesson: The north star and the metric tree