Why most message testing is complete BS (and how to fix it)

Is your message testing just throwing random ingredients into a pot and hoping it tastes good? In this no-holds-barred episode, the boys tackle the chaos of message testing and why most PMMs are doing it wrong. From useless A/B tests with tiny sample sizes to executives demanding "just one more tweak," we dive into what actually works. Tune in for laughs, practical frameworks, and why your mom's cooking method isn't a valid messaging strategy. We're covering:

Only 30% of product marketers have ever actually tested a message (and most of them did it wrong)

Why A/B testing is the "let's not commit" cop-out of marketing teams
The one-variable rule: why testing multiple things at once is destroying your results
The critical difference between message testing and A/B testing that most marketers miss
How to build a message testing framework that gets leadership buy-in immediately

If you've ever sat through a meeting where someone suggested "Let's A/B test it" for a campaign going to 150 people, this episode will either validate your frustration or completely change how you work. Either way, your next messaging project just got a whole lot more effective.

Timestamps

01:39 Question: How Many PMMs Actually Test Messages?

02:31 Why A/B Testing Is Often the Wrong Approach

04:00 Statistical Significance and Sample Size Problems

05:58 Testing Text vs. Testing Visuals

06:05 Messaging as Strategy vs. Tactical Implementation

09:01 Differentiating Message Testing from A/B Testing

12:15 The Commitment Problem in Marketing Decision-Making

16:16 How Trade Shows Can Validate Positioning and Messaging

19:45 Message Testing to Define Product Uniqueness

22:00 Eric's Process Misstep: Jumping to Experimentation Too Soon

27:45 The Scientific Method in Message Testing