Last year, I made a meme that very few understood. But I feel my fellow #RWRI participants will appreciate it 😅

Building LLM systems felt more like praying to the AI gods than engineering. But all that changed when I learned about eval-driven development. This Thursday (9/25) at 11AM CST, I’ll be sitting down with @HamelHusain for a live podcast to discuss evaluation frameworks and failure analysis in LLM systems. We’ll cover: - What's an AI eval—and why it matters? - How to run error analysis for open-ended tasks - Should you rely on LLM judges or code-based evals? - How to debug and improve your system once you have evals in place ... and more Join us live and ask Hamel your AI eval questions! 👉 Register here: luma.com/7izvlf3r

Building software on top of (unpredictable) LLMs can feel more like praying to the ML gods than engineering 🙏 But it doesn’t have to be that way… In a recent blog post, I showed how error analysis resolves the guesswork by giving us a systematic way to identify and improve the most significant failures from an LLM system. 👉 Check it out here: medium.com/@shawhin/how-to-i… -- Shout out to @HamelHusain and @sh_reya, who showed me the light when it comes to LLM error analysis in their fantastic AI evals course :)

I just received 3 baguettes from Sathish on @buymeacoffee! buymeacoffee.com/shawhint/c/…

3) Building (rather than buying) just because I can Before, I had a tendency to avoid paid software tools/services when I felt I could implement them myself. Now, I bias toward paying for solutions that can help me move faster.

1) Progress = learning more tech skills Going from a good to a great coder won’t move the needle as much as going from bad to mediocre in key non-tech skills (i.e., sales, marketing, writing, customer calls, etc.)

Technical founders love to code, but that alone won't make you successful. Here are 15 things I had to learn to become a (data) entrepreneur... 1. Talk to customers 2. Identify pain points 3. Sell 4. Sell to many people (i.e. marketing) 5. Make a website 6. Build a landing page 7. Write hooks 8. Write blogs 9. Post content 10. Cold outbound 11. Respond to emails faster 12. Use AI tools effectively 13. Look Foolish 14. Constantly learn 15. Keep going What would you add?

Last month, I booked my most expensive mentorship call yet at $700/hr 😬 While this might sound like a big number, the person on the other side of the call makes $100k/mo as a solo AI entrepreneur. Here are my key takeaways from this conversation... 1) Focus on customer pain points and problems, not your past experience and skills. 2) Price for the customer's value, not your time. 3) The bigger the price tag, the longer the copy (i.e. delay the ask) 4) Get hyper-clear on the customer and their problem, then attract them with free content (on socials) and a lead magnet. 5) The difference between 10k/mo and 100k/mo is more often business strategy rather than technical expertise. Although these points might be Business 101 for some, many technical founders (like me) never learned these basics. This is great news because it means I can probably get significant gains from a few simple adjustments. Super grateful to @jxnlco for taking the time to share his wisdom with me 🙏