AI in disguise: AI-generated ads outperform human-made ads if they don’t look like AI

About

2023 was the year of generative AI. Its advance sparked the imagination about the role of generative AI in creative industries in general and advertising in particular, and raised the question whether it can actually disrupt the advertising industry. Answering this question is nearly impossible using individual A/B tests as these only provide a snapshot into specific application contexts, which are subject to the researchers’ discretion. To address this question, we partner with a leading online ad platform, which released a generative AI-powered ad maker in 2023, to obtain a dataset covering more than 2 million daily ad-level observations across various industries. This dataset includes both human-made and AI-generated ads in a quasi-experimental setting.

In our quasi-experimental setting of ads that were created by the same advertiser at the same time, and as part of the same campaign but vary in terms of the usage of AI, we find that the click-through rates (CTRs) are higher if advertisers employ AI to generate advertising images, but not necessarily for the ad caption. Interestingly, this superiority of AI images holds only if the AI-generated images do not look like AI. We find that AI images are more likely to include a human face, but the presence of a human face tends to disguise the ad as being generated by AI. Our findings provide evidence that advertisers can benefit from leveraging mass-market generative AI tools to boost their online ad performance.

Jochen Hartmann’s research is situated at the intersection of digital marketing and machine learning, with a focus on the analysis of unstructured data (computer vision, natural language processing) and generative artificial intelligence (AI). His substantive research interests encompass social media, multimodal digital advertising, algorithmic fairness, diversity in advertising, and human-machine interactions. His work has been published in journals such as Proceedings of the National Academy of Sciences (PNAS), Journal of Marketing Research, Journal of Consumer Psychology, and International Journal of Research in Marketing (among others). His work on text and image mining is among the most read and most cited articles in high-ranking marketing journals. He is also a co-author of book chapters on machine learning and natural language processing.
In January 2023, Jochen Hartmann was appointed to the Professorship of Digital Marketing at TUM School of Management in Munich. Prior to that, he was an assistant professor at the Faculty of Economics at the University of Groningen. He earned his doctoral degree from the University of Hamburg. Before his doctorate, he worked as a management consultant at McKinsey & Company. Jochen Hartmann is a regular visiting scholar at Columbia Business School and a lecturer in machine learning at Mannheim Business School and Vienna University of Economics and Business. Jochen Hartmann is a member of the DFG-funded scientific network “AI algorithms for decision-making in businesses and organizations”.