Rafael Quintero 🍍

Auditing is fundamentally a marketing and relationship game. Everyone offers the same service: they look at your code and give you a PDF. Yes, of course there is skill variation, but past a certain point, it’s very hard for outsiders to tell one firm’s skill level from another. I don’t think even I could reliably compare some firms’ capabilities, and I’d like to think I know more than the average customer. Even if multiple firms audit the same commit hash, not every firm categorizes “high” and “medium” the same way. Similarly, the auditor assigned to the audit might have been having a bad week, etc. Since ranking audit firm capabilities is extremely difficult (maybe even impossible due to apples-to-apples limitations), outsiders must resort to judging by other metrics. In “the marketing game” — firms try to emphasize the “alternative metrics” they have an advantage in and are easy to understand: - “more eyes look at the codebase” (audit contest) - “we pay a higher % to our auditors” (contract-driven firms) - “we put out banger content, it proves we know what we are talking about” - “a founder is active on X and posts helpful content” - “our team wins famous CTFs” - “we specialize in topic [X] — that makes us better at it” - “we assign more auditors to job than other firms” - “we catch bugs others missed” (heard that one a lot, but was it really a “bug”?) - “we maintain an open source tool you already use” - “codebases we audit have never been hacked” - “we’ve audited brands you recognize” These claims can be true, but they don’t guarantee the audit is of high quality — or the best deal you could get for the quality. Even if the firm is good, and the auditors assigned to the job are good, they could have a bad week. The metrics above are probably correlated with quality, but they don’t guarantee quality. Fundamentally, those are narratives. Firms must tell a story that translates to “you can trust us to do a good job, even if it’s hard for you to evaluate the quality of our output.” You can of course increase your chances of getting audit deals by building relationships with potential customers. Again, since they can’t accurately measure the quality of your work, they’ll resort to “easier metrics” such as “do I like this person and do we have a good history?” Again — by no means am I saying “ignore quality, go all in on marketing.” But when you are in an industry where the customer cannot easily judge the quality of work — especially the potential of work that hasn’t been done yet — then the narrative becomes extremely important. Economists call this “differentiation” — how companies selling essentially the same product try to convince the customer they are different. For “differentiation” to work, it must be believable. “We work harder than others” isn’t believable because everyone says it. “We specialize in [X]” is believable because it’s easy to verify — but again — it doesn’t automatically mean the audit is actually better. I’m also not saying auditing is a bad business to be in — the profit margins are very nice and demand is rather reliable. There is a cultural expectation that code is supposed to be audited. This props up demand. I’m just telling you the reality of the game as I see it. If you have a hard time seeing this, consider the hypothetical situation where you are hiring a lawyer to help you deal with some legal issue. When talking to multiple firms, they’re all going to tell you they dealt with similar issues in the past or specialize in the area you are a dealing with. You could try to look up their past cases, but you aren’t really in a position to judge if they won easy cases or hard cases, for example. You’re probably going to pick a lawyer either based on a recommendation from a friend or based on the vibes they give you, and to a certain extent, how good a story they tell on their website. The skill of the lawyer matters, but it’s hard to judge it past a certain point. Same for auditing.

We made a smart contract dev framework for Vyper. We made a pretty cool feature. We think it would be cool in foundry. We made a ticket for it in foundry. Open source is amazing. github.com/foundry-rs/foundr…