A Methodical Approach to Testing New Media Partners
Sally Chi is a Growth Manager at Chime, one of the largest fintech neobanks in the US. Sally started her career working at ad networks, learning the ropes of user acquisition from the vendor side. Shifting gears a few years later, Sally was fortunate enough to join the Nexon team, where she dedicated her time to working on the Marvel franchise and other global game launches. At Chime, Sally manages the team focused on portfolio diversification, including the mobile program, affiliate & content marketing, web display and programmatic, and direct mail.
Learn more about Mobile Hero Sally Chi.
My job at Chime is to work on diversifying the portfolio of our paid media. Since joining more than two years ago, I have spent a lot of time speaking to partners, advocating for Chime’s products, and crafting tests to set both parties for success. To ensure Chime had long-term partnerships, I maintained an organized, systematic, and disciplined testing approach.
As an app marketer, finding reliable media partners you can reach new users on is one of the biggest challenges. But with a diversified plan, you will spread out marketing risk and set yourself up for future success. Here are the steps, tips, and tricks I learned along the way.
1. Do the Research
Always do your research before agreeing to work with a partner. Although this is time-consuming, it is worth the effort. I recommend doing this in a two-fold method:
Research on Your Own Time
There is plenty of knowledge within the industry. It is a guarantee that no partner has been untouched within your network of growth marketers. Chat with peers and leaders through Slack groups or ask for recommendations from industry friends. I promise your MMP representatives might even have an opinion or two.
Take a look also at industry benchmark reports. For example, Appsflyer, Singular, and other MMPs do a comprehensive analysis on metrics such as reach, retention, and in-app events across many partners. Utilize these informative reports every year.
Research Your Partner
The best way to get information is to communicate to the potential partner directly. Here are good conversation-starting points:
- Product & inventory: What is the actual product? Are you able to show an example placement? Has this inventory been tested before, or is it completely new? Is your audience a good demographic fit for my product?
- Opportunity size: How scalable is your product? How much do top partners spend on the platform per month?
- Targeting: What are your targeting parameters? Do you offer lookalike audiences? Can you target based on my events?
- Capability: Are you technologically capable? Can you connect to my MMP? Can you pass the cost?
- Protection: What is the level of reporting and transparency? What granularity can you report on? What about fraud—are you blocking in real-time or after the event has already happened?
- Product usability: Is your product self-serve or managed? How long does it take to get out of the learning phase? Do you have any unique tactics or strategies?
- Competition: Are my competitors using your service?
2. Plan Your Time
After working at big corporations and smaller startups, I learned that onboarding a partner always takes time to go through legal and compliance considerations. If your team expects you to launch a partner immediately, get some sort of timeline on when this could happen, depending on paperwork processing and contract reviews.
Secondly, understand the time investment to test a partner thoroughly. If they require time to get ramped up, make sure they set expectations upfront. If you need optimizations on events that mature in an extended cohort, then communicate with your partner so they can plan out how to spread your budget over a certain period.
If you are testing many partners at once, I recommend putting start/end dates into your calendar to track the timeline of your tests.
3. Establish goals, benchmarks and KPIs beforehand
You need to figure out your goals (objectives), benchmarks, and KPIs before establishing a test.
Goals: Determine what you are testing with your partner. Your goals should be quantifiable and measurable. From there, figure out how many conversions you will need to be statistically significant.
Benchmarks: Always have something to measure your results against, whether it is your best-performing channels or organic numbers. Depending on if you are testing a similar partner or not, you can determine a conversion rate which will help you figure out how many days it will take for you to get to statistically significant conversions.
Targeting: As for targeting, make sure you target your current demographic as broadly as possible. We are a banking app, so we make sure our targeting excludes anyone under 18 years old. You also want to make sure you know how your MMP or partner can break down the campaign into tangible results. If they can provide granularity data on each site ID, you might not need to launch multiple campaigns. However, suppose your MMP cannot send as much information. In that case, I recommend breaking down the campaign into groups such as interests, behaviors, or placements that make sense for your product so you can track the results on your end.
4. Talk to Your Partners
Your partners are not just lever pullers and button pushers. There is a high chance that your account manager’s goals align with your results. After working on the ad network side for several years, I cannot stress how often I wish I had more information from my clients. I recommend giving an overview of your product, how you came up with your KPIs and metrics, and how you established your test. Sharing as much data with them is also helpful. Throughout the test, keep your partner informed of what you see on your end.
5. Evaluate Test Results
The evaluation is based on a mixture of quality and volume. Most advertisers make their decision this way.
However, even if a partner does not hit your success metrics, ask yourself the following questions:
- Were they proactively working towards your goals? Was your goal within range if the partner was not given a time constraint?
- Were they a good partner? Did they closely communicate things to you?
- Do they offer an inventory that is unique and valuable that you cannot get anywhere else?
- Is my competitor aggressive on this platform? If so, is it important that I am also on there to be first to market?
Through my experience at Chime, I have found these five methodical steps crucial in helping me test and evaluate new partners. Preparation and research have always been key in setting new media partners and me for success. Remember to ask questions and have transparent communication—this will help establish trust and a long-term partnership.