For several of my direct marketing consulting clients, testing is the straw that breaks the camel’s back. Many marketing groups are stretched to the breaking point. The reason that happens so much is a topic for another discussion.

Just getting out the work takes everything they’ve got.

In such environments, the first thing to go is strategy. It is like walking on one leg. I would call that limping, not walking.

The title of an article I wrote “It Works Compared to What?” is a quote of my initial response to a marketer who tells me an execution works but does not validate it through ongoing testing. They may think they are walking, but they are really hobbling when they could be running.

You can download the pdf by going to the articles section on my web site at www.dmcgresults.com.

Marketing program results, regardless of size, should always be measured and evaluated for effectiveness, including return on investment. Not to do so is just inept.

As far as A/B testing goes, that really depends upon the actual marketing program and the size of the mailing.

In a B2C mailing of 100,000, you would certainly want to do an A/B test before doing the actual full-scale mailing.

However, in very highly targeted B2B mailings, the actual target audience can be quite small . . . 1,000, 2,000, 3,000, etc. In cases like that, even if you were to do a very small test, the results obtained might not even be statistically valid. Understandably, it might not be meaningful to test very tiny mailings — but the results should always be evaluated, regardless of size or scale.

– David Fideler
Concord Communications and Design
http://www.concordcd.com/

Marketing is often measured on product management and corporate marketing activities - positioning of the product, organization and timing of the product release, and how spectacular the booth looks at the big tradeshow.

If more marketing departments were measured on things like quality of leads, prospective customer response, etc. I imagine you’d find more direct marketing and testing.

It would be interesting to survey companies and see if there’s a correlation to the use of direct marking tactics and testing to Marketing being graded in significant part to revenue performance.

My experience with marketing managers is that they watch the profit numbers. For example, a newspaper made $4000 in ad profits in July 2006. In June 2007 the ad reps try a new marketing seminar. July 2007 shows a $5000 profit - up $1000 from last year.

If there weren’t other obvious influencing factors, the manager would assume that the seminar was successful. I haven’t seen a lot of scientific method involved in management in marketing, but I think their “gut” evaluation methods have merit.