14 August 2024

Fully automated AI vs. hybrid control: a performance comparison

In summary

• Most digital advertising platforms are now offering AI-based optimisation solutions
• Louder has done some testing on Performance Max search campaigns to find the optimal balance of AI and manual control
• Despite AI advancements, some form of manual control is still essential for optimising ad campaigns

With many advertising platforms now adopting AI based optimisation solutions, marketers need to understand the benefits and perhaps, some of the limitations, that these options provide. Many are asking the question - is AI really better than manual optimisation?

So, we decided to run a test to understand whether AI can run alone or is better used in conjunction with manual intervention. In this article, we explore the performance results of campaigns that use both AI and manual inputs from a human marketing specialist. We examine this across two different use case scenarios - Performance Max (PMax) and a search campaign operating with a full automated smart bid strategy.

Performance Max

The test campaign

Performance Max (PMax) is a campaign type in Google Ads that leverages Google’s machine learning to optimise ad delivery across all of Google’s channels including YouTube, Display, Search, Discover, Gmail, and Maps. Unlike traditional search campaigns, PMax does not require manual keyword input. Instead, Google’s algorithm automatically matches ads to relevant search queries so it can encompass branded, generic and competitor terms.

Our experiment aims to evaluate PMax’s brand performance in the following scenarios across four campaigns:

• Brand terms: Comparing the performance of brand terms driven by PMax versus traditional search campaigns.
• Brand term impact: Determining if PMax performance is predominantly driven by brand terms.

The results

All four campaigns performed exceptionally well with above average Return on Ad Spend (ROAS) compared to the overall account. Further investigation into the search queries, revealed that three of the campaigns delivered over 85% of conversions from brand queries, while the remaining campaign obtained 55% of conversions from brand queries. To provide context on its significance, it is important to note that the number of conversions exceeded 200 per month per campaign.

The next step was to identify if the brand was performing better than our conventional search campaign. We found that:

• CPC and CPA were on average, 2.5x more expensive in PMax than in the search campaign.
• While the ROAS for PMax was above average compared to the overall account, it did not perform as well as the ROAS for the brand search campaigns.
• The volume on the brand search campaign decreased when we started running the PMax campaign. However, when we began brand exclusion on PMax, the volume on the brand search campaign increased again.

Ultimately, brand performance was not exceptionally strong across PMax. We found that running brand keywords through traditional search campaigns was far more efficient. However, based on our findings, performance improved after applying brand exclusion to the PMax campaigns, effectively filtering out brand traffic. And this was only determined and actioned by the marketer.

Automated bid strategy

The test campaign

This experiment involves using an existing search campaign currently optimised with a Target ROAS bid strategy. Unlike PMax, where no keyword input is required from advertisers, search campaigns use advertiser provided keywords.

Louder has been running a Target ROAS bid strategy with manual controls on particular search campaigns for over a year. We decided to test the impact of removing these manual controls to allow the bid strategy to optimise itself. In this instance, the control refers to a human marketer setting minimum and maximum bid limits. Without this control, the algorithm is free to bid as much as necessary to win auctions likely to achieve the target ROAS.

Our testing approach was to remove the bid limits, allowing the algorithm to take full control of the bids without threshold. We allowed a six-week learning and optimisation period before making comparisons. No other changes were made to the campaigns associated with the bid strategy.

The results

During this six week period, reporting showed a 98% increase in cost but conversions increased by only 1%, leading to a 69% decrease in ROAS.

Below is an example UI displaying the performance results.

By removing the bid limits, the CPC increased by 122% driving up costs without improving impressions or CTR. Google’s explanation? The bid strategy is designed to explore different bid thresholds to optimise performance, which resulted in the inflated CPC.

While this strategy is expected to optimise towards the target after the learning period, it doesn’t appear to be achieving this. The search query report revealed that the CPC for certain queries surged to $50, despite the account’s average CPC being $2 to $3. It’s generally not best practise to spend $50 on a single click in order to gain just one potential conversion, unless within categories such as insurance or law which generally see high CPCs.

Also, as Google mentioned, consumer behaviour in the “messy middle” can be unpredictable, so if this click was missed, the user could still have converted with at a lower CPC, at a different time.

Based on this data, we concluded that it is far more efficient to run bid strategies with bid limits. Once again, illustrating the importance of human and manual intervention.

Louder’s recommendation

AI is undeniably essential for optimisation. It’s clear that bid strategies with auction-time bidding excel at optimising at the search query level, a task beyond human capability. Performance Max offers a promising approach for optimising at the audience level throughout their journey across multiple Google inventory streams.

However, we don’t believe it’s time to let AI take full control over campaigns without manual oversight. Based on the experiments, it does not seem to be achieving the most efficient results, so it is still necessary to maintain some manual control and intervention to achieve optimal performance.

Overall, we suggest excluding brand terms across PMax campaigns and setting bid limits for your bid strategy across search campaigns for increased performance.

Need more help?

Get in touch to discuss how we can assist you or your business and sign up to our newsletter to receive the latest industry updates straight in your inbox.