
How to Run a Paid Media Pilot: What to Measure and When to Expect Wins
A paid media pilot is a deliberately scoped test that lets you de-risk a decision before you commit real budget to it. You are not trying to prove that paid media works. You are trying to find out whether this campaign type, this creative approach, or this audience is worth funding at scale.
Two questions come up in almost every one of these conversations we have with clients:
- What should we be watching?
- How quickly will we know if it worked?
Both have answers. They just depend on what you set out to test in the first place.
Why Run a Pilot Instead of a Full Rollout?
You should run a pilot instead of jumping straight to a full rollout because a pilot buys you information at a fraction of the cost of a mistake. You are putting a slice of budget behind a question so that the answer, good or bad, arrives before you have committed the entire program to it.
That framing matters for how you judge the result. Good stewardship of a client’s budget is about avoiding and controlling where missteps can happen. A pilot that fails at 10% of spend is a stewardship win, not a loss: it kept the other 90% out of harm’s way.
Start With the Objective, Not the Tactic
Every pilot needs to start with a purpose. Where they fall flat is in trying to do too much: one test built to satisfy sales, finance, and leadership at once, with no consistent KPI to judge it by. Pick one goal, anchor it to a single source of truth for success, and get stakeholder buy-in on both before anything launches.
Think of it the way you would think of a science experiment. A clear hypothesis shapes the entire methodology. If your hypothesis is “a new campaign type will generate more leads at a similar or better cost per acquisition,” then lead volume and CPA are your North Star metrics, and every other number you look at is context. Whatever you write down at the beginning trickles all the way through the rest of the pilot, including how you decide it went. Get vague here and you will end up arguing about the results instead of acting on them.
How Much Budget Should a Paid Media Pilot Get?
A good starting point is around 10% of the monthly or period budget. That number can move depending on the business, but it is a reasonable place to begin the conversation.
The other route is a straight A/B split: 50/50 against your existing approach, or 60/40 if you want to stay conservative. Either structure works. What matters is whether each side of the test has enough budget, enough time, and enough data to produce a real answer, and that takes forecasting before launch. Look at expected CPCs, conversion rates, and volume for the specific platform and vertical you’re testing in, then work out whether your budget can realistically reach a meaningful number of conversions in the window you’ve set. A 10% slice that works for an e-commerce brand on Google Search may not come close for a MedTech brand on LinkedIn. If the forecast says you can’t get there, adjust the budget or the timeline before you start. Otherwise you haven’t run a pilot, you’ve spent 10% of your budget on a rounding error.
What to Ask Before You Launch
Run through these before anything goes live:
- What decision will this pilot help us make?
- What is the hypothesis, stated in one sentence?
- Which one or two KPIs define success, and what does “good” look like numerically?
- How long is our typical purchase cycle, from first ad exposure to closed revenue?
- How much budget can we commit, and is it enough to reach a conclusive result?
- How segmented is the test, and are we cutting the audience too thin to learn anything?
- What is our tolerance for a performance dip while the test runs?
- What would make us stop this test early?
Which Metrics Matter and When?
Your KPIs are the ones you will ultimately be held to: cost per lead, cost per acquisition, ROAS. But those are rarely the first thing you can see, so it helps to think in three layers.
Leading indicators show up first, usually inside the opening couple of weeks. In search, that is click-through rate and quality score. Stronger numbers here suggest your ad copy and content are resonating, and that your keywords, ads, and landing pages are lining up with what people are actually searching for. These will not close the case, but they tell you whether you are pointed in the right direction while the primary data is still filling in.
Mid-funnel indicators are usually the primary conversion you are tracking: lead volume, cost per lead, conversion rate, and lead-to-close rate. How long these take depends on the size of the purchase and how early in the journey you are meeting the customer. An upper-funnel campaign will take longer to show a mid-funnel result than a lower-funnel one.
Lagging business metrics are weeks or months out. Customer lifetime value, full-funnel blended ROAS, repeat purchase behavior. These are the numbers that tell you whether an expensive first acquisition was actually a good one. If a customer comes back, the CPA you flinched at in week two looks very different in month six.
If the pilot is awareness-focused, the whole set shifts. Impressions, reach, and frequency tell you whether you’re reaching more people without showing the same ad to the same person too many times. Beyond reach, look at whether people are actually paying attention. For video, that’s completion rate. For campaigns that drive traffic, it’s time on site. Those tell you more about an awareness pilot than conversion rate will.
What Is the Minimum Volume You Need Before You Trust the Data?
For Google Search, my rule of thumb is roughly 30 to 50 conversions over a 30-day period as an absolute floor. That is the minimum to feel reasonably good that the data is telling you something real, and businesses with higher volume should hold themselves to more.
It is important to set that threshold at the start of the pilot, not at the end. The point is to be able to say, “We need at least this many conversions before we call anything, so we know we are looking at something repeatable and not a strong week or a seasonal bump.”
How Quickly Should You Expect Wins From Your Paid Media Pilot?
When it comes to expecting wins from your paid media pilot, it depends on what you are testing. Here is roughly how I think about the tiers:
| What You Are Testing | Realistic Read Window | What to Watch First |
| Ad copy and creative assets | 1 to 2 weeks (one week minimum) | CTR, early engagement against previous copy |
| Keywords, campaign types, account structure | 2 to 4 weeks, ideally a full month | Quality score, CTR, conversion rate |
| Landing page and website funnel changes | At least a month, often 6 weeks or more | Conversion rate, drop-off by step |
| Audience targeting shifts for awareness | 4 weeks and up, longer for downstream effects | Reach, frequency, viewability rate, video completion rate, brand lift/recall |
Creative sits at the front of the funnel, so it reads fastest. With enough volume you can sometimes see in the first few days whether new copy is outperforming what it replaced, though I would still give it a week or two before acting.
Funnel and landing page tests need the most patience, and they are also where I would be most careful about scope. Rather than redesigning an entire page, test segments of it, and split traffic so you are not sending the whole pilot through an unproven experience. Do not break what is not broken.
Audience tests for awareness campaigns are the hardest to call. Bid strategies are often still learning, and the true mid-funnel and lagging metrics may never surface cleanly because the effort sits so high in the funnel. The fix is to decide before launch how long the test will run, and hold to it. Give it at least a full quarter so downstream effects have time to show up. In the meantime, judge the test on what it can actually move reach, frequency, video completion, and brand lift, along with early downstream signals like growth in branded search or direct traffic. A flat conversion rate in month one isn’t a reason to pause. It’s what an awareness campaign looks like at that stage. Nothing is worse than pulling a test because it looks flat, then watching it start to show up in the books two or three months later.
Should You Pivot in the Middle of a Paid Media Pilot?
You should usually give your primary indicators enough time before making any pivots to your paid media pilot. But if a leading indicator is clearly bad, say click-through rate well below anything you would expect, that is an early warning worth acting on. It may mean the test is not going to work, or it may mean something in the setup needs a fix, like imagery or copy that is not landing.
The risk is making so many changes before you reach statistical significance that you muddy the results and lose the thread of what you were testing. Then you are spinning your wheels with no clear path to the next decision. So no, you do not have to grit your teeth through an obviously broken test, especially when there is meaningful budget behind it. That is exactly why you watch the leading indicators even when they are not your North Star.
Know Your Purchase Cycle Before You Judge Results
This is the one that trips people up most often. You need to know how long it takes for someone to go from first seeing your ad to actually buying.
I had a client with two service lines. One booked same-day or next-day. The other booked out about a month. Every month they would come in alarmed that ROAS looked terrible, and every month, a month later, it turned out to be fine. The performance was never the problem. The measurement window was.
For MedTech, healthcare, and other privacy-first brands, that window can be considerably longer than what an e-commerce brand deals with. In some cases the gap between an ad click and a purchase decision stretches out for months, because there are approvals, referrals, and reimbursement processes that have to happen before anyone ends up with a device in their hands. None of that is a reason to avoid testing. It is a reason to agree on the timeline before the pilot launches, so nobody holds the campaign to an expectation the sales cycle makes impossible.
Do Not Judge a Paid Media Pilot on a Single Metric
Some businesses are locked onto one number, and I understand why. But a clean line from this click to this dollar is not always available to us. Sometimes the path is circuitous, and the honest answer is that attribution can only get you part of the way.
That is why I would rather look at a small set of metrics than one. Alongside CPA or ROAS, ask the broader question, “Did the business see growth in this area in the wake of the pilot?” The distinction between a booked appointment and a kept appointment, for instance, makes a real difference to what the pilot actually proved.
The Mistakes That Kill Paid Media Pilots Before They Start
The most common failures are not strategic, but rather structural:
- Not running the test long enough to reach significance
- Not funding it well enough to reach significance
- Hyper-segmenting the audience until there is nobody left to reach
- Holding the pilot to the same tight ROAS or CPL guardrails you use on a mature program, so the platforms never get room to find the connections they are good at finding
There are a lot of ways to cut the legs out from under a campaign before it has a chance to run. Committing 10% of your budget to something unproven can feel uncomfortable, but underfunding the test does not reduce your risk. It just guarantees you learn nothing. What you put in is what you get out.
And resist the pull of the last 24 hours. Monthly is the minimum useful scale for judging holistic performance. Nothing hurts more than making a real decision off a data set that was never big enough to support one.
Keep Running Pilots
One pilot is not the end of anything. Performance is iterative, and the next pilot is usually an iteration on the last one. The number of tests you run is the real lever, so do not get discouraged when one does not pan out. Set the hypothesis, fund it properly, give it room, and let it tell you something. Then go run the next one.


