We’ve run some A/B tests in the past, but they’ve mostly been superficial — headline tweaks, CTA colors, etc. I know it can be more powerful, especially for UX decisions, but I’m not sure how to structure those kinds of tests. Anyone have a solid framework?

You’re not alone — a lot of teams start with cosmetic tests and stop there. But A/B testing can go way deeper into UX if you frame it around behaviors rather than visuals. This breakdown on ab testing UX helped us rethink our approach.
Instead of just "which color converts better," it explores testing entire interaction models — like onboarding flows, form simplification, or layout hierarchy. The article walks through designing hypotheses, setting up valid control/test groups, and reading the data in a way that actually informs product decisions (not just boosts a metric).
We used it to test a multi-step sign-up vs. condensed single-screen, and the results surprised us — user friction dropped with more steps, not fewer. That’s the kind of insight A/B testing can reveal when it’s embedded into UX strategy instead of just marketing.