The Risk in Not Testing Your Direct Mail

 

Author: Alan Sherman, VP Marketing Strategy

If you’ve been in the direct marketing space for any amount of time, you’ve certainly heard about the importance of testing. But “testing” means different things to different people, and we see people approach it in a variety of ways. My colleague, Maggie Stack, recently published a blog on Direct Mail Testing. Consider this a prequel – insights into the risks of not testing.

Testing “Approaches” Run the Gamut

Some marketers don’t test on a consistent basis or simply don’t test at all. After mailing the same creative package, list or offer, for months or even years on end, they experience the inevitable performance decline, and think it may be time to test a new creative. Their creative team or agency offers a new creative that hopefully out-performs the old one and then the process repeats itself. Still other marketers occasionally test but are so risk averse that they test small creative attributes, such as a headline in a letter or on an envelope, which rarely moves the performance needle. And data testing happens less frequently than creative testing, even though data has a more significant impact on results.

A Fear of Testing

Why the reluctance to test? Some marketers simply don’t know how. Others may not want to risk the company’s budget on a test that may not out-perform the control. Sometimes it’s a combination of the two – “I don’t know how – so why take the chance?”

What About Data Testing? I hear less about this than creative testing.

Why is there so little data testing? Data can be tougher to test – nowadays it would be less likely for us to test an entire list. With the cost of postage, we must be as precise as possible. Segments or model groups within an audience are not always easily explained or understood. Many lists should not be mailed without the use of effective predictive models to rank prospects. It feels riskier than changing a creative headline, or maybe data is managed by another group in the organization.  Maybe the data testing setup and measurement can feel daunting. We can hold and touch creative – it feels more “real” than data and many of us like to think we intuitively know what works in creative when it’s sometimes best left to the direct mail creative experts. 😊 And yet data has a larger impact on response than creative.

The Risk in Not Testing Your Direct Mail

Some might apply the old saying “If it ain’t broke, don’t fix it” to direct mail testing. I would argue if it ain’t broke yet, it will be some day.  I would say that there is greater risk in NOT testing. Every creative package’s performance declines over time, and if you wait until that happens you could find yourself many months away from a new, high-performing package. The same applies to data. Doing nothing is short-term thinking that, over time, delivers inferior results. Regular testing is an investment that puts the long-term odds in your favor. Every test is an opportunity to learn and improve.

How frequent should testing be done?

Testing must be a regular ongoing process. Ideally, that means part of every campaign or at least testing quarterly. The aim is to develop a stable of high-performing creative packages and a precisely optimized audience by testing new data sources and predictive models to continually improve results.

We can make testing work for you. Testing is both art and science – Nahan offers both. We will continue to cover various aspects of testing in upcoming blogs.

 

Bio: Alan Sherman is our Vice President of Marketing Strategy. Alan enhances Nahan’s current value proposition with strategy solutions that support new/existing client relationships. For clients, he leverages market, customer, and competitive intelligence to build achievable strategies for omnichannel marketing success. His marketing plan strategies include targeted data, predictive analytics, testing, and creative that drive ongoing client performance improvement. In his spare time, Alan enjoys spending time with his family, traveling, going to concerts, watching sports (he’s a fan of the NY Giants, Boston Red Sox and Celtics), and walking the dog, even though it was just out.

 

 

 

Direct Marketing Strategy: Direct Mail Testing

Author: Maggie Stack, Account Director

At least once a week, while doing homework, one of my kids will say “when will I ever use this in real life?” I like to point out to them how often I use my Algebra skills, but I never thought I would use science in my marketing career. As my colleague, Alan Sherman, mentioned in a previous blog, we use direct mail testing to determine the best direct marketing strategy. This is where science comes in. By following the steps of the scientific method with a continuous improvement mindset, we aim to exceed our clients’ marketing goals.

Purpose

When thinking about our clients’ direct marketing strategy, our question is always how can we improve results. The exact Key Performance Indicator (KPI) we are trying to improve varies by client, but it always means a better Return on Marketing Investment (ROMI).

Research

We start by reviewing current marketing efforts. Who is the target audience? What motivates them to respond? What tactics are currently being used? How do those tactics work together?

Hypothesis

Once our research is complete, we make recommendations for the elements or options we believe will improve results. Sometimes this is the choice to reduce the cost of a campaign while maintaining response and sometimes it is a higher cost option that will improve response. This could be a new data source, a new offer positioning, a new direct mail format, or addition of a complimentary digital tactic. In direct marketing, the options are truly endless. Once we decide on what will be tested, we can begin the experiment!

Experiment

The two most commonly used experiments in direct mail testing are a split test and a multivariate test. Split tests, or A/B tests, involve testing the same package except for the specific element to be tested. Multivariate testing involves testing multiple elements at the same time. The right one to use is based on several factors: budget, available quantity, quantity needed for the result to be statistically significant, and the number of items we want to test. Once the experiment is in the hands of the prospects, we wait for results.

Analyze Data

Depending on what we are measuring, results could take months to gather. There must be enough responders to be confident in the results. Analysis comes in the form of charts and graphs. Our goal is to always improve results, but sometimes we learn what does not work. As my high school science teacher would say after a failed experiment, “if you learned something, the experiment wasn’t wasted.” In direct marketing, the win in a losing test is that it leads to better, more refined hypotheses.

Conclusion

While there is a conclusion to every individual experiment, direct mail testing should never end.  We believe marketers should always be striving to improve their data, improve their messaging, and improve the tactics they use. And we love partnering with those that feel that same. Looking to apply a little bit of the scientific method to your direct marketing strategy? We are here to help.

Bio: Maggie Stack is an Account Director with over 20 years of experience in marketing services and direct mail production. When she isn’t discussing data and creative with her clients, you can find her and her husband cheering on their children in hockey, baseball, and dance.