ad testing case study

AD TESTER USES PROPRIETARY A.I. TO LEARN WHAT USERS LIKE

There are millions of potential ad combinations on any given publisher website.  Every single ad has a measurable and definable affect on their ad earnings and users’ experience. We’ve found that all ads affect the user experience too.  The only way to know which ad combinations work for which user, is to test all possible combinations; which (for a medium sized site) would take a single human over 20 years to do.

Ezoic’s Ad Tester is an easy-to-use application for publishers that uses artificial intelligence (A.I.) to automatically tests all of the possible ad combinations on a site. It tests thousands of new ad combinations and learns from user reactions to determine optimal ad combination for each individual visitor.  The Ezoic Ad Tester application sits ‘on top’ of a publisher’s existing configuration and allows users to automatically optimize their site for both user experience metrics and ad earnings at the same time.

Ad Tester optimizes based on as multitude of important factors including  ad position, ad size , ad density ( in user session AND per page), ad proximity , ad type, and ad changes by landing page, geography, device size, browser, traffic source and time of day (this is not an exhaustive list). A feat unachievable by means other than Ad Tester.

WHY IT WORKS

Ad Tester is powered by an artificial intelligence system that learns and understands how all aspects of a user’s session affects everything else.  Simple site optimizations, like showing different ad combinations by traffic source (Facebook traffic or Google traffic should get bespoke ad treatments) and device size (iPhone 6 Plus or Galaxy 5s), have a huge effect on user metrics and publisher ad revenue.  Ezoic takes this several stages beyond that, by automatically configuring the publisher’s site to an optimal state of UX and ad earnings for each user, no matter what their profile.  Our aim is that each user gets a uniquely optimized experience.

WHY PUBLISHERS LOVE IT

Getting started is easy. All publishers have to do is integrate with the Ezoic platform and download our Chrome extension to start adding ‘Ezoic placeholders’ (potential ad positions) to their site where they’d like to test ads. Once these placeholders are added, Ezoic uses them as alternatives to begin testing.  Over time, the system find out which permutations of ads, by device, work better than others.  Some user groups might respond better to native ads, others to fewer ads early in a user session and more ads later in the session.  The data will feed the system to select the best configurations for each user who is visiting the site.

ad testing system
ad combinations

AD TESTER GOES FAR BEYOND “TESTING”

Trying to imagine an ad configuration that ‘suits everyone’ is almost impossible.  As the saying goes — you can’t please all the people all of the time.  Well with Ezoic — you can!   The Ezoic machine learning system tests different combinations of ad locations and sizes and measures the results.  When the data collected becomes statistically relevant / reliable, it uses it to make smart decisions about what will be shown to all users ‘like that’.   It does this on its own and automatically.  Over time, it improves the publisher’s site, and they don’t have to lift a finger — saving them time and making them more money. This is done continuously forever; even as behaviors change Ezoic continues to optimize the site.

POWER IS NOTHING WITHOUT CONTROL

Publishers maintain total control over where the ads display and the rules that are set about how many ads are shown per page, what types of ads and how much testing they want to do.  Ezoic is built to allow publishers to test Ad tester as much or as little as they like.  They get to choose the percentage of traffic that is sent through to testing; while letting the rest of their visitors experience the site as it was before.  The ‘original site’ acts as the control for UX comparison and session earnings comparisons (how much — per visitor — was earned, and how that stacks up against the original ad performance).

CASE STUDY ON INCREASED AD EARNINGS

Below are several cases in which Ezoic measurably improved both raw ad earnings and user experience metrics for publishers in completely different web niches.

Many ad professionals look at things like RPM and CPM as metrics to help them optimize their ad earnings. Through years of research, Ezoic has learned that neither RPM nor CPM metrics are particularly valuable when you are talking about measuring website monetization as a whole.

epmv case study

WHY ‘SESSION EARNINGS’ MATTER MORE THAN ‘RPM’

For example, let’s assume that RPM is revenue per page viewed.  If you were to change your site so that each visitor views twice as many pages, the amount earned per page would likely drop.  However, because you now get twice as many page views in total, the amount earned overall would be higher.  

So, in a situation like that, the RPM measurement would show a decrease in ‘performance’, even though the overall earnings are increasing.  If the publisher was monitoring earnings per 1,000 visitors (EPMV) it would show the increase in value and overall revenue increase.  EPMV (Earnings per [mille] thousand views) is the only metric that truly accounts for changes in all contributory measurements: CPC/CPM/RPM/PV/V/BR etc.   This is why Ezoic uses this measurement to track revenue improvements overall.  

You can see in the example above, that this site more than doubled its EPMV across all devices using Ezoic. This resulted in a net gain of millions for this website vs. their previous ad setup. This kind of growth is impossible without an artificial intelligence that is able to quickly test and learn from ad combination improvements.  Here are some common ways that Ad Tester impacts a publisher’s site…

USER EXPERIENCE & AD EARNINGS ARE IMPROVED BY AD TESTER

ad earnings affected by UX

One of the most important things to know about Ezoic’s intelligent ad testing application is that it thinks beyond ad revenue.  It ‘thinks’ rationally about maximizing a sites session earnings now and in the future without detracting from user experience.  This means making decisions and optimizations that affect both ad earnings and user experience based on goals that are set by the publisher.  If the publisher wants an equal weighting between UX and Revenue — that’s what the A.I. will deliver.  

Given the same demand sources (AdSense / ADX) and sufficient data and time to become statistically accurate, it cannot fail to deliver improvement vs a static ad set up — it’s a mathematical certainty.

You can see in the URL above that ad earnings increased on every device, including an increase of over 400% on desktop.  The site also saw a significant increase in returning visitors, a 9% decrease in bounce rate and a slight increase in time on site. Ultimately, this caused the site’s EPMV (session earnings) to nearly quadruple while they were using Ad Tester.

MOBILE MATTERS … A LOT

In the case of the URL, thefactsite.com, you can see that the publisher was able to increase revenue and performance across all devices, with a revenue increase in excess of 100% on mobile.  This is not unusual and is a regular occurrence for Ad Tester publishers.

Most site owners that spend time trying to optimize ad earnings on their site manually are doing so while working on it from a desktop console.  This often has the unintended consequence of having a site that isn’t as well optimized for ad revenue on mobile devices; which is a huge blind spot for publishers because mobile is the number one device size for site visitors and is still growing.

With the URL, thefactsite.com, you can clearly see that the platform’s impact on its mobile audience clearly offered a significant impact; by lowering bounce rates on mobile. A site can also win more traffic from search engines (because bounce backs for keyword search returns — back to Google.com — are believed to be ranking factor in Google’s search algorithm).

ALL SITES BENEFIT FROM AD TESTER

mobile ad earnings study

Even sites with experienced ad operations professionals that are responsible for increasing ‘ad yield’ can benefit immensely from Ad Tester.  This is because ‘better ads’ (i.e. improved demand in RTB or Header Bidding) can only go so far.  If the ad combinations are sub-optimal (i.e. static on all pages, no matter what the engagement level), there is nothing more that an ad operations professional can do.  Manual testing is impossible at this scale.  This is why AdOps teams have become huge fans of using the Ezoic platform to double, triple, and even quadruple ad revenue.

In the case of the site above, you can see where Ezoic has worked directly with the ad professionals on the site to ensure a balanced increase in revenue across devices. This resulted in lower bounce rates and much higher EPMV.

A.I. AD TESTING REQUIRES PATIENCE… BUT IT PAYS OFF

how much do publishers make?

It would be great if all sites could be optimized ‘in an instant’ for all pages, but unfortunately, testing takes time and is constantly moving target.  

Just like a human, the artificial intelligence that powers Ad Tester has to have data to make decisions.  Testing is not instantaneous and the system needs to absorb the results, crunch the data and optimize for future scenarios. It’s a continuous process that carries on as an organic feedback loop.  

There is no such thing as ‘the perfect ad combination’ – all users need the right ads for ‘their’ user session.  One ad (size) does NOT fit all!

The site above took well over a month before it started to realize the EPMV improvements from testing.  As testing continued, you can see that results slowly improved over the next 6 months; where EPMV has ultimately doubled.

Ad Tester is not meant to be a one-time tool that is used and discarded once a site has realized steady increases. User behavior changes regularly, devices change, browsers change and the engagement of your articles varies from page to page.  Sites benefit from continuous testing as it ensures the optimizations are steady and automatically respond to user behavior shifts or changes.

IT TAKES INTELLIGENCE TO FIND THE RIGHT BALANCE

mobile ad earnings case study

Ultimately, the variables that determine a publisher’s site performance are far too great to be tested or optimized efficiently by traditional methods. Ad Tester uses machine learning to understand important UX factors and ad combination data. It then leverages this data to make changes that enhance the site performance.   Even sites that are well put together and have been perhaps tested and optimized can benefit from the Ezoic artificial intelligence platform.

As you can see with the URL highlighted above, Ad Tester has the ability to powerfully impact sites that already believe they have a solid ad setup.  There is no such thing as the perfect site; there is always a way to boost ad revenue and improve UX metrics.  Ad Tester’s machine learning algorithm does this all for the publisher automatically, without the need to set up experiments or put in extra hours of work over time.

AD TESTER HELPS ENSURE POLICY COMPLIANCE

One of the most impactful elements of Ad Tester is that it exists on the Ezoic platform and automatically ensures that the site using the platform is in 100% compliance with Google ad policy guidelines. No one wants to spam out the users with too many ads (or indeed, leave money on the table because there were ad opportunities missed), but the temptation for many publishers can be to push monetization too far.  The benefit of using Ad Tester from a Google compliance perspective is that the rules are baked into the system.  Is having a 300x600 ad below the fold on mobile allowed?  Yes, then it will include that in the testing if the publisher has selected that ad size for testing.  A 300x250 above the fold on mobile for a Google ad?  It won’t show because it’s not compliant.  

This offers peace of mind for publishers and also offers Google a system that can enforce Google rules for the publisher’s benefit while not slacking off on the monetization.  This shows the tremendous impact on revenue and site performance that Ad Tester can have; while keeping publishers safe (sometimes from themselves!).

When a publisher is realizing unprecedented increases in revenue, the last thing any publisher wants is to see is a compliance issue that could compromise their entire business.

lycos goes mobile-friendly

The Lycos site’s are a great example of this. Lycos came to Ezoic as a recommendation from the Google Certified Publishing Partner team. They wanted to sort out compliance so that they could eventually ensure safe and steady monetization increases in the future.  They have millions of sub-sites in their portfolio and keeping them compliant for Google’s advertisers was becoming impossible.  They needed a scalable (technical) solution to help get the account back on track.

Ezoic stops malware at the DNS level, detects and excludes porn / unsafe content, excludes objectionable content (Google ads don’t show on pages with bad words on them) and places ads in acceptable locations for each screensize.

To date, Ezoic has ensured that over 1,000 of the Lycos sites are both mobile-friendly and Google ad policy compliant (this number is increasing daily).  It won’t be long before all their sites are running on Ezoic and staying safe with Google.  This gives the publisher the ability to utilize Ad Tester and other Ezoic apps on these pages to increase revenue and continue using Google ad products like AdSense & Ad Exchange.

A.I. FOR PUBLISHERS

It’s been a long road…   It took over 3 years to build the backend for Ad Tester (in stealth mode – with venture backing, working mostly on our own sites) and another 3 years to bring the front end features to market commercially.   

Ad Tester is now the world’s largest automated A.I. system for publishers.

Our system fulfills the incredibly hard task of working with any site, easily, in a way that is Google ad policy compliant, compatible with every CMS, and tested in a way that is consistent with the quality and performance that publishers expect from a Google Certified Publishing Partner.  It has to be able to learn from changes to Google Analytics metrics and from changes to each ad impression combination and make all necessary optimizations automatically.

The entire Ezoic team was involved in creating this innovation.  At its core, it’s about getting ads to fit the content.  

We had developers working alongside data scientists, ad professionals, publishers, and even a rocket scientist to build an artificially intelligent application that could perform this never-seen-before level of multivariate testing for our customers.

Ad Tester increased collaboration internally but also powerfully strengthened Ezoic’s relationship with Google. Look no further than the recent AdSense blog that highlighted a case study of customers using Ezoic’s Ad Tester. One of the subjects of the case study actually increased total ad revenue by more than 400%.

advanced-reporting-types

WORLD-BEATING ADVANCED REPORTING

Ad Tester gives Ezoic publishers access to reports that show them ad revenue alongside user metrics by landing page.  For the first time, publishers can know which pages make them the most money and can adapt their content creation strategy to fit with the user engagement fed back via Ezoic.   There is even UTM tracking to help understand which newsletters drive the most revenue or engage with an audience best.

AD NETWORK AND HEADER BIDDING MEDIATION

Ad Tester offers — not only dynamic ads for ADX and AdSense ad locations — but also allows all publishers to plug in their own ad networks and header bidding partners.  This is the only tool in the world to offer A.I. for ad placement with unlimited ad partner optimization as well.  Our bespoke, proprietary solution for Header Bidding (the Ezoic Header Bidding Mediation Suite), gives access to Ezoic to allow header bids from all the major header bidding companies — in one easy to manage place.

ad mediation study

THINKING BIG & MOVING QUICKLY

With all the publishers currently on the Ezoic platform, there have literally been trillions of ad combinations tried and tested by the artificial intelligence that powers Ad Tester. This has resulted in significant revenue increases and major site performance improvements for thousands of websites in less than a year’s time.  

Over 90% of publishers who try Ad Tester for more than 30 days stay with the system long term.  

Ezoic created Ad Tester as a sequel to its popular Layout Tester app.  Layout Tester operated similar to Ad Tester but also tested page layouts as well as opposed to strictly ad combination changes (many publishers had difficulty giving over control of their layout or had subjective issues with their sites would appear using this tool). This inspired Ezoic to move ahead and build Ad Tester – it’s what our publishers told us they wanted.

And it worked!  Many of the publishers who were concerned about the impact of Layout Tester on the personal design of their sites quickly took to Ad Tester and heralded it as a major turning point in their business.

“I’m using Ezoic, again. New features are a game-changer for website publishers using display ads (including AdSense).  I’m talking huge. Early results are great!” – Jon Dykstra ( fatstacksblog.com)

Ad Tester was implemented less than one year ago and thousands of publishers are using it today.  Hundreds of publishers join the platform each week and the result is millions in gained revenue for these publishers.

CHALLENGING THE STATUS QUO:

As Ad Tester grows in popularity, there is no limit to the amount of value it can add to the Google ecosystem.

Ad Tester is quickly becoming the flagship tool that drives publishers to try the Ezoic platform. Growth is fast and is expected to continue its acceleration. Our product represents an unprecedented innovation for publishers.  Ezoic is the real deal and is offering real change to every single one of our publishers.