Tim Davidson

If you're doing CRO and not running thorough retrospectives, you're losing 95% of the value of your tests. And it isn't hard to do. If you're on Intelligems or Shoplift, both have an API. Pull the numbers straight into Claude or ChatGPT and get it to summarise what actually moved. Did you hit significance? Did you reach your minimum detectable effect? All the statistical technicalities that tell you whether you're jumping at shadows. Then open Microsoft Clarity or or whatever analytics tool you're on. Just make sure it integrates with your testing tool so you can filter down to each group. Put the control and the variant side by side and play spot the difference, like you did as a kid with two pictures. Then use your creativity and what you already know about your customers to work out why the patterns changed. You'll get plenty of those guesses wrong. Doesn't matter. Over time they compound into real knowledge about your customers, and eventually the puzzle starts to look like something. Here's a real story to illustrate this point. We work with a brand that has a small number of SKUs. Their main product comes in 10 colours, all of them sitting on the collection page. That collection page has always had a really high conversion rate, but almost everyone ends up on the PDP anyway. I couldn't work out why. To me the collection page looked like pure friction. Through a series of tests it clicked. People were using the collection page to decide which colour they wanted, because seeing all 10 side by side beats clicking through them one at a time on the PDP. That has led to a run of experiments that have done really well for PDP performance. We know what customers are trying to do there now, so we can make it easier for them to do it. We would never have figured that out without retros. One more thing. Save your retrospectives somewhere Claude or ChatGPT can read them. Once you've got a few, you can ask the AI to find patterns across your audience that are hard to spot yourself. To be fair, the collection page theory was mine. Claude didn't hand me that one. It did surface pieces I'd have walked straight past. So if you're not doing retrospectives, take this as your wake up call. You can go back and run them on tests you've already finished, and even on changes you shipped with no data attached. Use data where you can get it. A feedback loop is worth far more than the raw numbers on their own.

Every CRO program eventually runs out of ammunition. Finding problems on a store is easy. Solutions are the hard part. We've come up with several answers to this, but the one I'm having fun with at the moment is our experiment library. I pulled together every experiment my team have run, then scraped the web to find lists from other CRO teams. I ended up buying a few more libraries just to increase the volume. After some heavy deduplication, weeks of adding example screenshots and filling in the details of each experiment, I'd made a library of 742 test ideas that not only am I super proud of, but one that is helping us become better at testing. Here's how I'm using it. Each experiment has a hypothesis, solution description, psychological targeting, and a series of other data points. Experiments are mostly sorted by page with a few broader categories for global experiments like sales, timers, etc. I ping Claude and ask for 10 experiments that would be worth running for a client. It pulls the previous experiment retrospectives, the client profile, their customer behaviour analysis, and a whole bunch of other context from my Obsidian vault, then figures out which experiments might be a good fit. Once I'm happy with the experiments, they're copied from this Airtable database into the client Airtable base, so there's no double handling. After the experiments conclude, the results are pushed backwards. I record how many times each experiment runs, and the aggregation of the results. My goal is to run as many of the experiments as I can and find the trends of which ones work for which clients. This isn't one of those typical "I've got a lead magnet, comment 'lead magnet' to get it" posts. I'm not giving this asset away, sorry. It's too valuable and specific to how we work. But I did want to share the story of how I built it, and how it's helping increase our velocity and improve our win rate, in case anyone was interested.

AI probably isn't killing SaaS. But for page builders, the writing is on the wall. Before the wave of LLMs arrived, tools like GemPages solved a legitimate problem for lots of Shopify brands. Liquid was tricky and Shopify had real limitations. Page builders let brands spin up landing pages and make changes without being handcuffed to a dev. Shopify has grown up since then. And AI tools now solve the same problem with far more flexibility. Every brand doing reasonably well that got into bed with a page builder is starting to ask the same question. How do we switch to Claude Code? Because the new workflow is hard to argue with. Load up your design system and Figma project, then ask Claude for a new landing page. You'll have 5 variations to pass around the team within minutes. Pick a winner and have Claude build it. With a bit of QA from someone who knows what they're doing, you've got a working template. No more dragging pixels around a GUI praying everything stays responsive. And the whole process packages into skills, so it's repeatable and the knowledge transfers across your team. The catch is that jumping ship isn't straightforward. Page builder residue gets everywhere, and the systems don't play together. If GemPages is embedded deep in your store, you have to purge it and rebuild from a clean baseline. Sloppy AI code is a real risk. But if you were using a page builder for landing pages and UI changes anyway, inline CSS packaged into a template is just as isolated from your theme as GemPages ever was. My prediction: brands that grew up on page builders will head back to clean, lightweight native themes and use AI for their enhancements. We're walking several brands through this exact off-boarding process right now. P.S. Keen to hear from anyone who disagrees. Is there a job page builders still do better?

I managed to sneak a look at another agency's CRO research. And my imposter syndrome disappeared. For context, it was a stock standard CRO audit. Analytics, heatmaps, a heuristic review and a stack of recommendations. Upfront: the researcher was genuinely good, and I don't want to be the guy who hangs sh*t on another agency for not doing things my way. 90% of the report was great. The presentation was miles better than anything I put out. I was immediately impressed. But the deeper I got, the more it read like someone who wasn't thinking hard about what they were looking at. Most people can open a heatmap and eventually work out what the data says. Lots of visitors interacting with the hero? Must be engaging. The harder questions sit underneath. Why are people dropping off where they are? Does the behaviour match the reason they came to the store in the first place? It made me reflect, because this is exactly how we used to work. Most of our focus went on the mechanics of the site. Finding friction, making buttons more obvious, that kind of stuff. Getting people from point A to point B. (Arguably still the biggest lever for most stores.) But the layer this report missed is who your customers are and what actually drives them to buy. Two examples from our own testing: A brand we recently started with kept failing every test that added persuasive content to the page. Turns out their customers are super visual. They don't read, and they push back on anything that feels like friction. We cut the page length and stripped almost all the written text. Conversion rate jumped around 14%. Another brand sells protective gear for contact sport athletes. Their customers are highly neurotic and want every scrap of information they can find. We doubled the page length and conversion rate went up. Same lever, opposite directions. Without the psychographic work we'd never have found either test. So if you've never done a psychographic analysis of your customer base, this is your reminder. P.S. If this makes sense but you've got no idea where to start, hit me up. I'll help you figure out what your customers are actually motivated by.

Testing fast creates problems slow brands never see. Those problems are a sign you're doing it right. We're on a mission to run 200 experiments per client, per year. I've been writing about some of the challenges that surface when you're moving that quickly. I didn't think much of this announcement at the time, but it caught the attention of a few folks at @Intelligems. They asked me onto their podcast to talk about "terminal test velocity". The concept is simple. Run experiments as fast as your traffic will physically allow. Then deal with what breaks. You burn through your idea backlog quicker than you can refill it. And tests start fighting over the same pages. Handling those problems well is the difference between an amateur operation and a mature testing program. On the episode, Alex and I get into: 1. What terminal test velocity actually means 2. How to never run out of good A/B test ideas 3. How to run more than one test on the same page 4. How to prioritise when the backlog gets out of hand The clip below is me and Alex talking through running multiple experiments on a single page. Cheers to Alex and the whole Intelligems crew for having me on. Full episode:

My best CRO experiments came from thinking I'd broken a client's store. Panic-checking the entire purchase journey, assuming everything is broken, finds friction nothing else does. Adrenaline is a great co-pilot for CRO.

How getting inside our customers' neurotic heads led to our biggest win this year. We work with a sports protection brand. They sell one piece of protective wear. Over the time we've worked together we've stripped a lot of friction out of their site. Steady gains, nothing dramatic. The conversion rate didn't really move until we ran a psychographic analysis of their customer base to understand what was actually driving people to buy. That's a lot of fancy words for something fairly simple. We scraped everything we could find. Thousands of Reddit threads, thousands of social posts, every review they had. We spoke to customers. We pulled the logs out of their customer support conversations. Then we analysed the lot to work out who these people are and why they buy. Two things came back. They're sceptical. Borderline neurotic about it. And security is the strongest driver. That's the whole reason the product is on their list. The scepticism was the interesting part. Most brands live in a bubble where you assume everyone knows who you are, how your product works and why you're special. Almost nobody does. Even the people who have heard of you aren't thinking about you. So the job was to cut through that and reassure them the product has been tested properly. We added a set of logos. The hospitals and medical organisations that have studied and backed the product, linked through to the actual studies. Those logos were already on the site. We moved them to the right spot and made them clickable. 15 minutes of design and dev time. Every metric in that funnel moved the right way. And we never would have found it without going deep on the customer first. Honestly the process is much easier now than it used to be. Drop everything you have about your customers into Claude or Codex and build a psychological model of what's driving them. There's a bit more to it, but not much more. If you haven't done it, start there. It'll sharpen your paid creative, your website copy and how you present your offers. Keen to hear from anyone else going down this path. P.S. The test is still running, but the composite metrics are all pointing in the right direction.

Can someone explain the value of letting AI write comments? Every time I post I get the same templated responses. I kind of get it. Commenting helps the algo see you're engaged. But don't you read back over the shit the AI wrote and cringe? When you see it writing "Great insight, so true, here's the thing you just said that stands out." Or the other one "This is the part that nobody's talking about." It feels like a high school essay where the word count is being filled in without saying anything. Dead internet theory is in full effect.

I recently started working with two new brands. We're now in the weird "new girlfriend" period. This goes for a few months while we're figuring out the chemistry. With our long-term partners, the approval process is basically gone. They trust us enough that we're just part of the team at this point. They bring us problems. We bring back solutions. That's the whole relationship at this stage. New clients don't have that yet, and they shouldn't. Anyone who's worked with an agency has wondered the same thing early on. Is this the agency that actually delivers, or the one taking your money and hoping nobody checks the numbers? My job right now is proving we're the first kind. You can't just tell someone you're reliable. It takes time, and it takes results. For CRO specifically, the first month can be a genuine flop. We're still learning the audience and what actually moves them. There's a real risk in that opening stretch. We come in confident, take our best shot, and sometimes we miss. Usually it evens out over a couple of months. A few wins land, the partner sees how fast we move, and things start to settle. By about six months in, the relationship changes. They come to us with questions instead of checking up on our work. It's a strange stretch to be in. Slightly nerve-wracking until we're through it. But that's when the real magic happens.

I A/B tested the exact same change in three different spots on a single PDP. Top: significant loss. Middle: flat. Bottom: Winner. The change was introducing the phrasing "peace of mind." We picked up on customers using that exact phrase, over and over, in their reviews. The idea was to repeat it back to them and make the messaging feel more relevant and tailored. What I didn't have a good read on was where to put it. So we tested all three spots: - top of the page, near the pricing info - middle, next to features and specs - bottom, above the reviews Top: -> -16.92% conversion rate -> 98.9% chance of losing to the control -> surpassed MDE Middle: -> -4.82% conversion rate -> 73.4% chance of losing to the control -> didn't reach MDE, declared flat Bottom: -> +14.47% conversion rate -> 98.6% chance to beat the control -> surpassed MDE Same sentence, lost at the top, flat in the middle, won at the bottom. Here's my read on why. The top of a product page is doing a different job to the bottom. Up top, people are still figuring out what the thing is and whether they're in the right place. An emotional reassurance line there just reads as clutter. By the middle, they've accepted what it is. They're working through how it actually functions, so it still doesn't land. By the bottom, they've done that thinking. They're looking for a reason to trust the decision they're about to make. That's exactly where "peace of mind" belongs. I didn't plan the test around that theory. I only worked it out after looking at the three results side by side. Felt a bit silly once I saw it. It's obvious in hindsight. If you're building a product page, the same message doesn't work everywhere on it. Different visitors are at different points in the funnel depending on how far they've scrolled. Put the right idea in the wrong spot and it gets in the way.

1/ Run enough A/B tests and your Figma file stops making sense. Some experiments are still being designed. Some are already live. Some just got killed. That's three versions of "the website" existing at once, constantly. 2/ No system for that and it gets messy fast. Someone opens the wrong file. The dev builds off stale designs. Everyone starts pointing fingers. Velocity drops. Bugs creep in. 3/ Our fix: one live version of the site in Figma, always up to date. My designers weren't thrilled at first. It's real manual work. But it's the reason we can run tests at the speed we do now. 4/ Second rule: annotate every interactive element. Design can look identical while the functionality behind a button changes underneath it. Skip the note and the next person merges that change away. Nobody notices until it's live. 5/ If you're running a lot of experiments, build this now. It's way harder to retrofit later. What does your team use to keep designs in sync with live tests?

LinkedIn's corporate warriors keep announcing they've solved the world with AI. For me it's done something less heroic. It's let me stay lazy and still do a better job. Here's what I mean. I run a CRO agency. Five of us, a pile of clients, and we're always running experiments. At the start of every period we set a few big themes we're chasing. Simplify the educational content when a product is too complex for people to get. Strip the friction out of a mobile experience so nobody has to stop and think. At the end of each experiment I run a retrospective. Did it move the needle, and what did we learn. The problem is scale. We're a small team, everyone wears five hats, and a lot of the analysis lands on me. Picking apart 15 to 20 experiments a week used to eat a huge chunk of my time. I pride myself on being lazy. I'll always take the shortest path that still clears a decent bar. So I'd go through the heatmaps and session recordings, look at the test, draw my conclusions and move on. Quick. Honestly, sometimes not as thorough as it should have been. These days I can be just as lazy and get a much clearer result. Here's how it works. I've got skills that pull the data straight from the Intelligems API, or scrape Shoplift with a little tool and hand me clean JSON to read. The metrics are already sitting there. I do the part that can't be automated: watch the heatmaps and recordings, talk out what I think customers are doing as a voice note, give the test a heuristic once-over. Fifteen minutes, maybe less. Then I chuck the lot into Claude and let it compile the write-up. My Obsidian vault holds how I speak and write, so it spits the doc out in plain Australian English that's easy to read and doesn't sound like a robot. We work out what the experiment actually taught us. Sometimes it nails it. Sometimes it's wrong and I set it straight. The difference is I can now clear 20 experiments in a few hours instead of half my week. I can even hand it to the guys on my team without it turning into a time sink. So no, AI hasn't made me better or fixed the world. It's just let me stay lazy. Which, if you know me, is the highest praise I've got.

One of the biggest problems we're solving at Clean Commit this year: how to A/B test a landing page without wrecking your ad metrics. Here's the thing most people miss. When Meta learns your landing page, it optimises for how customers react to it and drives traffic accordingly. Then you run a test. You change a bunch of variables and the algorithm gets confused. Your cost per impression and cost per click climb. Even if the page itself performs fine, the ad metrics behind it usually tank. There's no real way around that on a traditional testing platform. The only workaround we've found: spin up one landing page per variant and marry each to its own campaign. Control page, one campaign. Variant page, another campaign. Want more variants? More campaigns. Then you pull the numbers from each and work out the result by hand. It's not apples to apples. Different visitor counts, different impressions. But it's the closest thing to a fair test, because the algorithm gets to optimise for each page on its own. The bonus is you see how the page performs and how the campaign performs. That matters, because CRO doesn't happen in a vacuum. We rarely work with a growing brand that gets less than 50-60% of its traffic from paid. Paid is the way forward whether we like it or not. So if you're only testing page performance and ignoring what happens to your campaigns, you're looking at half the equation. It's a big problem. We're building a process around it now, and at some point we'll probably turn it into a product. How are you handling this? Curious how others are combining their testing tools to solve it.

Last year our average A/B win rate was >20%. This year one AI workflow has pushed us to >30% We started scraping every piece of public data we could find about our clients and their customers to build a full psychographic model. Once we laid out customer motivations into twelve buckets, the random patterns we'd been analysing suddenly made sense. I honestly think this is the biggest unlock for any team doing CRO that's trying to get into the mind of their customers. The best part is that it's fully automated. All you need is an Apify account and one Claude Skill. I've built a full step-by-step guide that anyone can work through in 3 minutes. Want a copy? DM me "Customer Research".

I used to love writing about how we made our clients $150,000+ in additional revenue from one test. As far as I knew, this was true. But as I've read deeper into revenue projections from people who know more than I do, I realise that some of these numbers were just wrong. My calculations were based on the revenue gain numbers in @shoplift and @intelligems. If the winning experiment ran for a month, hit significance and reported a revenue gain of $20,000, I'd simply multiply this by 12 months in the year and declare that I'd added an extra $240,000. But every store we worked with has ups and downs to their performance. Maybe we had a big win during an "up" month and the rest of the year would be down. Or maybe the effect of the changes would diminish over time. There's too many variables to draw a clear correlation. Here's a good example. One of our client's Meta account was compromised which cost them months of algorithm optimization. We'd recently run a great experiment for them that increased their PDP conversion rate by 22%. According to the metrics, this would have been over $300k in additional revenue. But their Shopify account told a very different story. Their sales dropped by more than $300k within 6 months. That's why these days I'm more reserved about making big dollar amount claims. To be safe, I'll usually just declare how much revenue was made by the experiment during the time it was running, because that's the only number we have confidence in... even if it doesn't look as good on social media.

My team's BIG goal for 2026 is terminal A/B testing velocity. We get paid for winning tests, so running as many as possible makes sense. The concept: run as many experiments as our client's stores can support. But there's a ceiling because of Shopify's architecture. A store can only support a single theme and each page can only run one template at a time. Tools like Intelligems and Shoplift let you run multiple themes or templates as tests, but each parallel test dilutes the traffic, reduces the experiment's power and threatens its chance to reach statistical significance. Our solution is two parts: 1. Schedule tests with extreme precision 2. Pick an A/B testing tool that supports randomized participation so we can run parallel tests Point #2 took me a really long time to figure out. The idea is that a single visitor can be part of multiple experiments at the same time — and as long as they're not influencing each other (like nested components), the interaction rate between experiments won't cause inaccuracy. Since implementing this approach, we've gone from running 4-5 tests a month to 12+. My personal goal is to run >200 experiments per client, per year. Why bother running so many? Apart from the fact that CRO is ultimately a numbers game, we work with some clients on a pure performance basis. We only get paid when experiments win. Super high velocity is also one of those things that's easy to point to when clients ask "what makes you different?" Would love to hear from anyone else running a CRO program that's been down this path.

Every time someone posts "here's a mistake most agencies make," they're telling on themselves. That's a mistake they were making. They just assume everyone else was too. You can talk about your screw-ups in public. People respect that. What they don't respect is pretending you've always had it figured out and everyone else is behind.

I spent last week deep in Sephora's $16.8B funnel Here's what smaller brands can steal right now Sephora dominates the beauty industry. Their app has over 18 million actively monthly users. Their loyalty program is industry-leading. and their landing pages are a masterclass in CRO tactics that: - Fuel desire - Drive conversion - Encourage upsells Once you know how Sephora makes it work, the formula is simple. In this breakdown, I analyse: - Meta ads - SEO/GEO - Tech stack - Checkout flow - Website funnel - Digital strategy - Homepage audit - Product page audit - Collection page audit - and more BONUS: I've recorded a 10-minute video to explain some of the more complex techniques. Want the breakdown? Reply and I'll send it over

Unpopular fact: 4 of 5 website changes hurt your store and only the top 5% of brands do anything about it. 80% of A/B experiments fail to increase revenue. There's only 5% of online brands actively running A/B tests. Which means everyone else is losing. Just because you're not measuring changes doesn't mean your store is magically immune. The thing is that it's not a sexy topic so it's easy to ignore. Conversion rate people love to talk about the big wins. "We added a trillion dollars with this one test". And this is great, but the REAL value in a CRO program is loss prevention. To keep ahead of competition, every growing brand: - Add products - Update offers - Publish content - Refresh designs - Try out new plugins - Expand navigation items - Introduce loyalty programs Most change are likely to hurt more than they help. If your store has more than 50,000 monthly sessions This is your sign to implement a CRO system. The first step is to audit your store to make sure you've handled the easy stuff first. Get my 148-audit CRO checklist here: cleancommit.io/resources/sho…

I spent over two weeks breaking down a conversion funnel worth $13B+ Nike earns more than entire countries. But how do they actually convert customers? I went deep on figuring out Nike's CRO secrets. In this breakdown, I analyse: - SEO - Meta ads - Tech stack - Checkout flow - Website funnel - Digital strategy - Homepage audit - Membership flow - Product page audit - Collection page audit - and more Want the breakdown? Reply "Nike" and I'll send it over