Measure a Mumbai creator campaign’s lift
Mumbai, Maharashtra · Fictional practice case
The renewal deadline
Mumbai footwear brand Nivara had used creator videos to launch a monsoon collection. Two days before its next production run, growth lead Aditi Sen had to decide whether to renew the same creators, renegotiate their fee or pause the programme. The partnership lead cited 180 last-click orders. Finance asked whether those shoppers would have bought without the videos.
The brand had previously judged creators on discount codes and landing-page traffic. That rewarded creators whose followers purchased immediately, but it also rewarded whoever reached shoppers closest to buying. For this pilot, the growth team secured randomized eligibility for paid creator amplification. Organic creator posts remained public, so control users could still find them.
Three views of performance
The agency reported creator-attributed orders in its own dashboard. The experimentation team used all users assigned to treatment or holdout and counted orders after refunds. Finance used ₹550 contribution per retained order after delivery and payment costs and before the ₹75,000 creator fee. Aditi needed to reconcile these views before deciding what to put in the next budget.
A smaller test was commercially feasible. The agency would discuss a ₹50,000 renewal if Nivara focused on two creators and provided ready-made product footage. The catch was that the new campaign might reach fewer people, so the earlier experiment could not be treated as a fixed forecast.
Exhibits
| Measure | Eligible for paid creator promotion | Holdout |
|---|---|---|
| Assigned users | 10,000 | 10,000 |
| Retained orders | 460 | 350 |
| Observed rate | 4.60% | 3.50% |
| Measure | Value |
|---|---|
| Creator fee | ₹75,000 |
| Contribution per incremental retained order | ₹550 |
| Orders credited by last click | 180 |
| Proposed smaller renewal fee | ₹50,000 |
Discussion questions
- Reconcile last-click attribution with the randomized estimate.
- Find the lift required to cover ₹75,000 and the break-even volume under the proposed ₹50,000 renewal.
- Recommend an action. What can the randomized comparison say about paid amplification versus the whole creator programme?
Teaching note
Reveal the visual and solution
Estimate the effect of assignment
Compare 4.6% and 3.5% across all assigned users. The 1.1 percentage point difference implies 110 incremental retained orders per 10,000 eligible users. The 180 last-click orders answer a credit-allocation question. Using them to calculate incremental profit would imply ₹24,000 of gain, whereas the randomized estimate implies a ₹14,500 loss.
Separate statistical uncertainty from break-even
The standard error of the conversion-rate difference is about 0.28 percentage points. An approximate 95% interval for lift is 0.55 to 1.65 percentage points, equivalent to about 55–165 orders at this scale. The full programme needs 137 orders, or 1.37 percentage points, to cover the fee. The point estimate misses this threshold, but the interval spans it.
Set the terms of a smaller test
At ₹50,000 the hurdle is 91 retained incremental orders. Renegotiate only if the two-creator plan can plausibly deliver that volume. Predefine exposure and refund windows, set a holdout and record organic spillover. The experiment estimates the effect of making users eligible for paid amplification in the presence of public content. It cannot isolate the value of all creator activity.