You Need "Actions" To Take Action

Some­one once asked me, “How many vis­i­tors do I need to deter­mine the valid­ity of my split-​​testing?” For exam­ple, if you’re split-​​testing two dif­fer­ent head­lines, how much traf­fic do you need to make a good judg­ment call on which head­line is the real winner?

That’s an inter­est­ing ques­tion, and one with no real answer, other than “a lot.” And there’s a spe­cific rea­son why…

Gary Hal­bert once said you need 40 actions. John Reese says 200.

Per­son­ally, I like 100. You see, both Hal­bert and John are right, since Gary is refer­ring mostly to offline direct mail and John to online marketing.

Offline, peo­ple receive mail and open it. They read it, and then they decide to take action.

There’s a delay, and a plethora of cir­cum­stances you can nail down, depend­ing on how tar­geted your mail­ing list is, and how you decide to mail out your piece.

But online, so many vari­ables come into play.

Online, peo­ple react more quickly than to offline direct mail because there’s no delay. And traf­fic can come from so many sources, with so many out­side influ­ences that are far harder to control.

John Reese said some­thing on a tele­sem­i­nar recently that’s well-​​said. On a coach­ing call with Johnathan Mizel, we were talk­ing about, for exam­ple, Taguchi and multi-​​variable test­ing. John men­tioned you need 1,000 actions with Taguchi to make it really sta­tis­ti­cally significant.

What he said, and what I agree to, is this…

Taguchi was cre­ated for the man­u­fac­tur­ing indus­try. The dif­fer­ence between split-​​testing man­u­fac­tur­ing parts ver­sus mar­ket­ing ads (and espe­cially online mar­ket­ing) is that, in man­u­fac­tur­ing, you’re deal­ing with phys­i­cal com­po­nents (like car parts, since Taguchi was orig­i­nally cre­ated for the car industry).

Cars and car parts are fixed, inan­i­mate and pre­dictable.

But in mar­ket­ing, you’re deal­ing with peo­ple… you’re deal­ing with psy­chol­ogy… which is indi­vid­ual, unpre­dictable and sus­cep­ti­ble to change depend­ing on so many out­side influ­ences.

So if you get 40 actions this week, you may make a judg­ment call. But it may have been influ­enced by:

• when peo­ple saw your ad,
• where they saw it,
• the mood they were in,
• the search they conducted,
• the mind­set they were in,
• the spe­cific web­sites they saw prior to view­ing yours,
• what brought them to your web­site (was it an endorse­ment email? an affil­i­ate link? a sim­ple search? an adwords ad? did they know about you before see­ing your ad? did they see it in the morn­ing or at night? Mon­day or Friday?)…

… And so on.

All of which can change dra­mat­i­cally, online, with each and every result, each and every vis­i­tor and each and every test. And thus skew your results from test to test.

For exam­ple, 40 results this week may prove dif­fer­ent sta­tis­tics than 40 results next week.

(I’ve seen this. I made copy­writ­ing deci­sions based on small split-​​test results. But when tested and tracked again, the sta­tis­tics were com­pletely dif­fer­ent. Even with Taguchi!)

So, in my esti­ma­tion, 100 is safe. It’s what I’ve per­son­ally been using in my test­ing. Is it per­fect? Not at all. The more, the merrier.

The more traf­fic you gen­er­ate, and the more sales or actions you pro­duce, the bet­ter the result, the more sta­tis­ti­cally sig­nif­i­cant the test and the lesser the mar­gin of error.

But 100 is more sig­nif­i­cant with a thinnner mar­gin of error than with, say, 40 results. The bot­tom line is, really, that you’re best to go with “actions” or “results,” whether that’s sales, opt-​​ins or click­throughs. Not vis­i­tors. (NEVER go with just “traffic.”)

About the Author

Last 5 Posts By Michel Fortin

• How to Capture and Captivate Attention
• The Real Problem With The Flaw of Attraction
• Two Shining Stars in an Age of Darkness
• Changes, Relaunches, and Extreme Makeovers
• Superior Value Equals Superior Sales

Other Related Posts

• Put Your Copy To The Test
• Write Magnetic Headlines With These 7 Tips
• Three Tips To Raise Your Response
• Three Tips For Magnetizing Your Copy
• The Seven Deadly Sins of Website Copy