Testing is a key building block of any paid media campaign. It is all about controlling risk, giving you the ability to make data-informed decisions to maximise your output.
Human instinct is usually to ask, what if it goes wrong? But what we need to be asking is, what if it goes right? By building a testing hypothesis, you will understand what it is you are aiming to achieve through this test. That will show you the potential if the test goes right.
If you think about all the marketing channels and campaigns you now run, once upon a time these were brand new. You hadn’t yet dipped your toe into their waters, but at some point you took that leap either as an early adopter or follower. These channels have proven to be a success, and there is more potential for your business out there to discover through testing.
You need to move away from the mentality and defence of ‘this is the way we’ve always done it’, and ‘because the results are good’. Because although things may be working right now, without testing how do you know things can’t be better? As with anything in paid media, things become fatigued over time and you risk your performance stagnating or declining just by maintaining the status quo. Nothing is static.
The Importance of Testing
Testing is crucial to driving growth and performance increments through any of your campaign activity. Whatever the channel there are always improvements that could be made via testing.
Testing also helps you research and learn more about your audience. It can help you to test different approaches to aspects such as creative and messaging. This gives you insight into what resonates with each of your audiences. Because their reality is different, audiences will become customers of your business for different reasons.
Finally, it helps you to evaluate your data and make better decisions to understand the viability of new approaches and channels.
What are the Barriers to Testing?
The nature of testing means that you do not know for sure what the outcome will be. That can increase the barriers to businesses being willing to run tests.
The biggest barriers we see are:
- Struggling to understand and decide on what to test
- The impact on revenue if the test doesn’t work out
- The size of marketing budgets
- The impact of algorithms and automation
- How to draw useful conclusions from the tests
What are the Risks of Not Experimenting?
If you aren’t currently testing or are averse to testing, be sure you understand the risks of not doing so.
The biggest risk that you face are:
- It gives the competition an opportunity to get ahead. That could be in either new or existing channels. Right now, in the current markets, there are unprecedented levels of competition. It is crucial you are testing to at least keep up, but more importantly to get ahead.
- You miss opportunities to get in front of your customers. It is important you are keeping abreast of how your customers are interacting online, and this involves testing new channels. Buying cycles are expanding so it is essential you aren’t missing these potential touch points.
- Eventually without testing your performance will first stagnate and then decline. Things are constantly developing and evolving. For example, what resonated with your audience 6 months ago may no longer work as effectively. Or platforms that were popular with a particular audience two years ago, may well have shifted.
What is the Difference Between Campaign Testing and Optimisation?
Although they both aim to improve performance, there are some key differences between testing and optimisation. Optimisation is all about small incremental changes to improve campaign performance. It could be something like adding a negative keyword, or applying bid adjustments for a certain device.
Testing, on the other hand, is about creating or reshaping through experimenting with specific elements. Testing is more defined, it has allocated budgets, a set time frame and a clear hypothesis of the desired outcome. Testing is planned in, whereas optimisation happens at any given time and can often adjust multiple elements at once based on the data available at the time.
Types of Paid Media Testing?
When it comes to paid media and campaign testing there are different approaches that you can take.
a/b testing
- a/b tests allow for controlled experiments that split traffic and delivery. You can usually define the split, but most often it is 50/50
- You select 1 element that you wish to test so it is clearly defined and compare the results to determine which is best
Example of an a/b ad test setup in Google Ads is below:
Multi-variant campaign testing
- This is all about testing multiple elements and variables all in one go. These variables can span multiple factors such as ads and landing pages
- Due to the nature of the tests, you require more data to determine any statistical significance
- It can also be harder to analyse the results, where possible we would recommend sticking with a/b tests if you are new to testing
New approaches
- In itself, not a testing type, but it is a mechanism for testing
- These experiments run in full and are not split-tested, but you can apply approaches to reduce the risk
- These tests could be the launch of a new channel or campaign type
- Typically, they would require additional budget in order to run the test
- Because they are brand new, there is no baseline in performance
How to Successfully Run Campaign Testing
Experiments should not be just a shot in the dark. They should always be informed and backed by data. Before you launch any campaign test or experiment you need to build out your hypothesis.
- Clearly define what elements you are going to test
- Establish how these elements align with your overall business and marketing objectives
- Understand what success looks like and what you are trying to achieve from the test
- Be clear on how you are going to measure and analyse the results
- Determine when the test will run and how long it will run for. Factor in how performance differs for seasonality and also days of the week. Always set the start date for the test in the future (at least the next working day), so that the test is not skewed by half a day’s data
- Set your budgets
Which Campaign Elements Can You Test?
When building out your testing hypotheses, prioritise your tests based on the likely impact of the results. You should prioritise your most meaningful experiments.
- Channels
- Messaging
- Creative
- Landing pages
- Bidding
- Targeting and audiences
- Offers and pricing
- Calls to action
- Lead forms
In the example below John Lewis have kept the text element of the ad as the control, and are testing the visual creative. This makes it easy to determine which ad is more successful. This can be expanded out and tested against multiple audiences, each of which might show a different outcome.
There are endless amounts of tests that you can run. One of our recent tests for example involved testing Google Automation in ads against our own control. We have been testing the difference in performance when using responsive search ads with all assets submitted, or just the minimum required in pinned positions. The test is to see if our controlled ads work better than the dynamic ads that are generated into different variations at each ad auction.
Our results so far have been fantastic in some of our accounts with an uplift in both CTR and conversion rate.
Budgeting for Tests
To get the most out of your tests it is recommended to have a core budget, that should achieve 100% of your marketing goals and a test budget. Anything that is achieved through the test budget is then icing on the cake.
Typically, a test budget accounts for between 5-20% of your total budget, but in order to make a test worthwhile, so you aren’t pumping money into a test with no statistical significance, you should assess and determine the budget required.
Tracking and Measurement of Campaign Tests
You need to ensure you have sufficient tracking to determine the success of any test, and learn from the experiments:
- In platform
- Google Analytics
- Call tracking
- CRM
- Offline vs online data
- Reporting dashboards
Why the Structure and Scale of Testing Matter
When analysing your data, you need to consider the context and what this means at both the micro and macro levels.
For example, looking at 3 different ads within one ad group:
- There will be multiple components and variables
- You are only looking at the ad in one scenario
- You don’t know which element is the cause of performance
- There could an element that is holding the best ad back
You need to review data on a small and large scale to determine its true performance. An ad that performs well for one keyword or audience, may not perform at the same rate across a whole account.
Naming conventions are also a part of this, as these are key for reporting and analysis purposes. Names that don’t make sense or aren’t easily searchable make it hard to run reports. You should always aim to give logical and descriptive names across all your activity, even outside of testing.
Think about what information helps to define your test. Consider including things like who the audience is and the creative used. You can develop abbreviations that help you to define and pick these elements out. You can also number and keep a record of these in a data sheet. All of this makes it easier when running data analysis. You can filter and pivot data based on these.
New vs. Proven Channels to Test
You can use specialist tools to help you identify new channels where you could reach your audience. These could be brand-new emerging channels that offer huge opportunities for your business as an early adopter. Or it could be channels that you can see your competitors are already exploiting.
The important aspect of any new channel is its ability to reach your desired audience. Audiences within each channel are changing all the time. The age range on Tik Tok for example is progressing to include those beyond the 18-24 scope.
Campaign Testing Conclusion
Ultimately, if you aren’t using testing to give yourself proven performance data that informs your campaign decisions, then you are likely leaving a lot on the table. If you want to discuss how you can get started with testing, the best place to start if you aren’t sure is with an account audit. Get in touch today to book yours in.
