Attribution evaluation, cross device switches and offline campaigns
In the last post I described how to develop data driven attribution models. I also explained how to calculate more realistic CPO and ROMI on the basis of these models as the basis for optimizing digital marketing budgets. Evaluating whether re-allocations of marketing budgets on the basis of data driven attribution models actually works is not trivial. It’s not easily possible to ab-test one budget allocation or media plan against another. What could work is testing the previous budget allocation based on last-click attribution CPO in one region and the new one based on data driven attribution modeling CPO in a similar region. For Germany this could e.g. be Munich against Hamburg. But it’s still not a valid ab-test.
A better approach is to make the required changes to the budget allocation. This would then be treated as an intervention and methods of causal inference, to measure whether there was a causal impact, would be applied. I won’t go into the details as they can be read about in this blog post. But ultimately we would use macro-level data (instead of micro-level user-id data) and treat the budget optimization similar to something like a major product release or PR campaign. We would then do time-series analysis to see whether there was a significant deviation from what we would have the time-series forecasted to be.
As I’ve already mentioned in the first post of this series, an interesting problem with doing user journey analysis and attribution modeling on a cookie basis are device switches (e.g. researched on iPad, but bought on desktop) and cookie deletions. Device switches can be dealt with as soon as we have something like a user login to relate different devices to the same user. This should be taken care for by an integrated tracking solution. For example at Jimdo we know that when a user works on her website from her desktop computer and later logs in through the iPhone App that it’s the same user and that there is just one user journey instead of two.
Not properly accounting for device switches could lead to over- and under-estimating the effects of specific channels. Take the following example: there is a mobile campaign on Facebook that never seems to bring in any new sales. This could be because most of the users that saw the mobile Facebook campaign (didn’t sign up, no user login) eventually bought the product on another device. We would underestimate the performance of this particular mobile campaign and overestimate the effect of campaigns that the user might have interacted with before the sale on the same device that she bought the product on.
The macro-data level perspective already described above could be of help here. We could test the impact of our mobile campaigns by reducing the ad spends in these channels and use causal inference to evaluate whether this “intervention” had a significant effect on our sales. If not, there probably wasn’t a significant cross-device effect in the first place.
Macro-level data analysis through time-series analysis and causal inference can also be used for estimating effects of channels where tracking data on a user or cookie basis is not possible. These channels could include tv (as written about here), print, radio or any other type of “offline” channel. For tv analysis GRPs and reach-metrics are being provided by the agency and should be included into the analysis.
Marketing Attribution System
Marketing decision makers require an integrated marketing attribution system. This would include tracking and analyzing all user journeys, developing data driven attribution models and calculating CPO and ROMI on this basis. It’s important that this system is independent of the marketing channels, one of the reasons I wouldn’t consider Google Analytics. Other reasons are, that GA’s interface is complicated, it isn’t transparent enough, nor is it exactly clear which data is considered (only converting journeys or all?).
The system would most likely be implemented as a SaaS solution and the UI would have to be simple and straight forward. There would be a section where the tracking snippets, which the user has to integrate into the different channels and campaigns, can be individualized. And then there would be a reporting front end that would display the performance indicators of the campaign, including costs, conversions, CPO and ROMI. A mockup of such a simple report can be seen in the header image of this post.
An optimal marketing attribution system would take into account user logins to reduce the risks posed by device switches. It would also integrate a macro-level perspective to allow analysis of offline channels and further insights in cross-device effects.