Why iOS 15 Changes Everything for Data Science Teams Using SKAdNetwork

In case any mobile marketer missed it, Apple announced the apocalypse at their Worldwide Developers Conference (WWDC) 2020.

The four horsemen of the apocalypse came in the form of an iOS device pop-up, or as Apple calls it, App Tracking Transparency (ATT). This pop-up sounded the death knell for the IDFA and mobile marketing on iOS as we knew it. The drip of updates to SKAdNetwork (SKAN) since WWDC 2020 offered some respite for mobile marketers, so there was some hope for Apple at WWDC 2021 to announce SKAN improvements as part of iOS 15.

Here are 4 of the most wished-for SKAdNetwork improvements on the wishlist of mobile marketers:

• Creative ID: Mobile marketers wish for some insights into which ad creatives drive quality installs
• Campaign IDs: Limited to SKAN’s 100 campaign IDs per ad network, mobile marketers and ad networks wish for more campaign granularity
• Web-to-app attribution: Mobile marketers wish for SKAdNetwork to support a web-to-app solution that would attribute users on iOS devices who ask not to be tracked via ATT
• Access to raw/unedited postbacks: Mobile marketers wish for access to raw/unedited SKAdNetwork postback data for auditing purposes—data which in early versions of SKAdNetwork Apple forwarded directly to ad networks responsible for driving installs

All four of these most-wished-for SKAdNetwork improvements have important use cases for mobile marketers. And any one of these improvements would have a significant, positive impact on how marketers measure and spend on iOS marketing campaigns.

Mobile marketers made their wishes known, WWDC 2021 came, and Apple delivered. At the 2021 edition conference, Apple announced support for one of these wished-for features, and, thankfully, it was probably the most impactful.

Outline

iOS 15 changes everything
1. Modeled conversions
Pre-iOS 15
Post-iOS 15
2. Consistent attribution reporting for modeling purposes
Pre-iOS 15
Post-iOS 15
Probabilistic attribution for actual revenue reporting
Campaign pROAS for performance reporting, optimization, and budget allocation
Media mix models for outputting optimal ad network-level budgets
3. Underlying performance of withheld campaign IDs
Bring on iOS 15

iOS 15 changes everything

As part of the announcement of iOS 15, Apple will allow advertisers to receive the raw postbacks from the device—i.e. the source of truth for mobile marketing. As Apple wrote in their updated documentation:

Access to this postback data beginning in iOS 15 for data science teams completely changes everything. Ever since the introduction of advertising attribution on mobile via third-party solutions, advertisers have relied on self-reporting by the largest ad platforms.

With iOS 15, mobile marketers can finally get the unedited attribution of their mobile marketing efforts. Furthermore, marketers don’t have to rely on ad networks to share attribution data with them. Soon, marketers will be able to simply collect it by themselves from the attribution source. This is huge. Although mobile marketers lost the granularity of user-level attribution with SKAdNetwork, they gain the benefit of being able to audit how their marketing dollars are spent across ad platforms in a way they never could before.

SKAdNetwork is a cross-ad network, deterministic attribution solution, which means data science teams can compare how advertising works in an apples-to-apples approach across all mobile ad networks and platforms.

Let’s delve into the key benefits of how raw/unedited SKAdNetwork postback data aids the efforts of data science teams focused on measuring and optimizing mobile performance marketing investments.

1. Modeled conversions

Pre-iOS 15

As part of their support for SKAdNetwork, Facebook and Google both announced that they’d be modeling missing data from ad campaigns where they received incomplete reporting from the SKAdNetwork API due to Apple’s privacy threshold.

Apple’s privacy threshold is designed to protect user privacy, limiting the reporting of the Conversion Value and source app ID when it falls below some unknown level. There’s much theory about how this threshold works, but ultimately it means that the ad network doesn’t receive a Conversion Value for every install counted.

To help advertisers’ data science teams fill in the gaps, Facebook and Google use their extensive data science capabilities to model the missing Conversion Values. The majority of other ad networks decided to share the postback data they receive from the SKAdnetwork API in a raw format and without alteration. Facebook and Google model what Conversion Values they should get credit for and report only this data to the advertiser. This is a controversial approach, as this reporting, in turn, dictates how budgets are allocated by advertisers because the Conversion Value is the key piece of data that determines how well an iOS ad campaign is performing.

Post-iOS 15

Starting in iOS 15, ad network-reported modeled conversions are at best a directional piece of information for an advertiser’s data science team who previously had to rely on modeled conversions as the only source of truth.

If a data science team wants to model the unknown Conversion Values, they can now do so starting in iOS 15, using the unaltered postback reporting from the SKAdNetwork API. The benefit here for data science teams is that the modeling approach can be consistent across ad networks and they can ensure there’s no bias toward any single ad network by over-under reporting Conversion Values. This means every ad network competes for budget on a level playing field within the rules Apple set in SKAdNetwork.

2. Consistent attribution reporting for modeling purposes

Pre-iOS 15

Reporting for SKAdNetwork campaigns has been inconsistent at best across mobile ad networks. Because of how Facebook and Google report SKAdNetwork postbacks, it’s impossible to compare the performance from all campaigns and ad networks. This has resulted in mobile marketers using app-level blended return on ad spend (ROAS) as a key performance metric. Blended ROAS is a measurement of the total revenue generated in the app by blending revenue from both paid and organic traffic sources.

Related Article: All (LTV) Models Are Wrong, but Some Are Useful

But blended ROAS is a dangerous metric to use. This metric doesn’t give any insights into how individual campaigns or ad networks are performing. Although optimizing blended ROAS is a problem that should be undertaken by data science teams to achieve the optimal media mix model, it’s not the most pressing problem given they can’t even compare ad network performance to determine where to spend budget. A data science team unable to compare ad network ROAS makes it impossible for them to allocate the appropriate monthly ad network budgets, resulting in suboptimal portfolio spending.

Post-iOS 15

Consistent attribution reporting across all SKAdNetwork campaigns starting in iOS 15 would unlock significant opportunities for data science teams. At the most basic level, it allows for apples-to-apples comparisons of Conversion Value data across all SKAdNetwork campaigns. Data science teams shouldn’t stop there, however, as they can’t compare SKAdNetwork campaign performance to MMP-attributed campaigns on both iOS and Android.

Probabilistic attribution for actual revenue reporting

With the granular level of SKAdNetwork reporting available in iOS 15, data science teams will be able to probabilistically attribute SKAdNetwork campaign revenue down to individual users. This model (in Figure 1 below) would create a campaign membership probability that every install into the app belongs to every SKAdNetwork campaign (and similarly for organics). With this model, data science teams can tag a user with the campaign membership probability and the model then tracks that user as they generate revenue within the app. They can then report the daily actual revenue driven by an SKAdNetwork campaign as an install cohort matures.

Figure 1: A campaign membership model that creates a probability every app install belongs to every SKAdNetwork campaign (and organics). Receiving raw SKAdNetwork postbacks for Facebook and Google means data science teams can apply this model to those ad networks.

Campaign pROAS for performance reporting, optimization, and budget allocation

Most mobile marketers currently use SKAdNetwork to report the performance of D0 KPIs, whether that’s revenue, events completed, or some combination of both. Optimizing campaigns toward D0 metrics is diametrically opposed to how all app businesses are run from a corporate financial point of view. Most app businesses invest marketing dollars against some target payback window for when they’d like to see 100% ROAS of their investment—normally 180 days or more into the future. D0 KPIs don’t help here.

Raw SKAN postbacks in iOS 15 will allow for data science teams to apply a campaign pROAS model to Facebook and Google campaigns, allowing for consistent campaign performance reporting across all platforms and channels of paid UA. If data science teams use the previous campaign membership model to attribute predicted revenue (at the user level) rather than actual revenue, they then get a campaign pROAS for every SKAdNetwork campaign.

The diagram below is the simple mathematical model behind this. We multiply the predicted revenue at the user level by the campaign membership model probability to get a campaign predicted LTV for every install for every SKAdNetwork campaign ID. If we add up the predicted revenue contribution for each app install for each campaign, we get a campaign predicted revenue (LTV) for each SKAdnetwork campaign.

Today, AlgoLift by Vungle can output campaign pROAS for SKAN campaigns for all ad networks except for Facebook and Google. With consistent attribution reporting across all ad networks made possible by iOS 15, we can provide predictive ROAS reporting for SKAdNetwork- and MMP-tracked campaigns on iOS and Android, allowing mobile marketers to easily compare their paid user acquisition performance across platforms, ad networks, and campaigns.

Figure 2: Campaign pROAS reporting for SKAdNetwork campaigns.

Media mix models for outputting optimal ad network-level budgets

Without access to raw SKAdNetwork postbacks, it’s impossible for mobile marketers to allocate the optimal ad network budgets because they don’t know the post-install performance of campaigns. With iOS 15, data science teams can model the campaign pROAS of all campaigns and therefore aggregate this data to the ad network level to understand relative ad network pROAS performance. They are also able to understand the pROAS of organics using a probabilistic attribution model for predicted revenue. With both ad network pROAS and organic pROAS, data science teams can now build a model that outputs the optimal ad network budgets for Facebook and Google, ensuring each ad source receives the appropriate level of budget based on its current performance rather than some previous bias of historical performance.

3. Underlying performance of withheld campaign IDs

This one’s a little more challenging to unpack and mostly theoretical at this point. Both Facebook and Google hold back most of the 100 campaign IDs available in SKAdNetwork for their testing and campaign optimization, reporting performance only at an aggregated level against 8 (Facebook) or 9 (Google) of the campaign IDs.

With insights into the raw postbacks from SKAdNetwork beginning in iOS 15, data science teams will be able to look under the hood as to what Facebook and Google are doing with these “withheld” campaign IDs.

Having access to this data may help data science and mobile marketing teams find answers to the following questions:

1. Does access to this data change how to create campaigns?
2. Does access to this data help with ad creative testing and iteration?

Bring on iOS 15

Access to raw SKAdNetwork postbacks through iOS 15 is a game-changer for sophisticated data science teams. At AlgoLift by Vungle, our data scientists share that same excitement. We can’t wait to utilize this data for our customers to improve their SKAdNetwork campaign measurement, optimization, and budget allocation. As we learn more about iOS 15, we’ll be sure to share it in a follow-up blog post. So keep it locked to the Vungle blog.

Learn more about AlgoLift by Vungle’s measurement solutions that solve for Apple’s privacy changes by contacting us below.