Sundae Labs. I work on Cardano and Midnight.
Chicago, IL
Joined April 2012
- Tweets470
- Following1.1K
- Followers169
- Likes4.7K
I... I actually appreciate this perspective: inverting what we've all agreed is true ("slop is bad").
Relatedly, slop _is_ a signal that your codebase doesn't have strong enough constraints on agents. Whether that be mutation tests, lint rules, or even a norm that when a builder agent is done it needs to spawn a separate agent for detecting slop.
If the agent creates slop, it's because it was allowed to. This is where the community needs to be focusing: building tools that essentially throttle agents to the point of nearly only being able to do the "correct" (high-quality, non-slop) thing.
Slop is just a feature suggestion wrapped in code that nobody has any emotional attachment to. It's an executable idea. Who cares if the code needs to be rewritten. I love slop!
1
76
We're not yet at the point where devs who care about code quality are able to give free rein to agents.
It doesn't matter how good are your docs, skills, CLAUDE.md, etc. The agent, without being completely restrained, will make mistakes and write code that adds unnecessary complexity, doesn't align with what you want, departs from repo conventions and patterns, etc.
Until we have perfect formal verification that completely captures your idea of what the codebase should be...
You must interrogate the code.
If we’re not looking at the code what should we becoming better at?
I’m struggling to find a clear answer.
1
35
"Pin" is a dev verb that has invaded dev jargon. I never saw it or used it before and now I see it all over, even in non-slop prose.
This is actually an instance of a useful piece of jargon given to us by the AIs.
33
Updating skills and AGENTS.md files is insufficient, the agent will still break the rules and then say, "whoah whahoopsies, sowwy!"
Actual improvement comes from identifying what went wrong and then developing a *machine checkable* way to identify that class of wrong-thing. Now the agent can't break the rule without failing the check.
"Agent self improvement" is a really powerful concept.
In short: have agents review past work, identify where they went wrong, and propose updates to skills and AGENTS.md files to prevent future failures.
Here's a multi-agent system you can implement:
1. Have scorer agents review past conversations and assign pass/fail grades along metrics you define: code quality, efficiency, task compliance, etc. This can be a subagent swarm reviewing in batches, or run as an automation using hooks with your agent harness
2. Let self-improvement agents review conversations with "failing" scores, receiving both the raw transcript and a "report" generated by the scorer agent. The agent reviews those failures and creates a PR to update skills or AGENTS.md files, citing failed runs as evidence for changes
51
This misses the point. Before AI, when humans moved slowly, we had time to review code, to ensure that it actually does align with what we want.
We do not have that luxury with the volume of code that AI's are producing. And you can't have AI's do the review (you can, but it's unreliable) because the oracle of what the code should do is *in your head*, so by definition the AI must guess.
I have such a hard time with the AI code slop discourse.
Is code quality a real concern? Absolutely.
In 20 years of working on codebases with other humans, did I ever feel like we had a real handle on code quality? No.
It feels easier to get everyone's agents to understand and follow conventions and best practices than it ever has been to get a bunch of humans to do the same.
3
74
e2e and smoke tests are the best oracles to release agents on to start a long-running project.
Unit tests are best for specific edge-cases to come back to after the "silver bullet" has given you an e2e proof of something working.
30
Build me a harness that works with me to translate my request into a formal lang (like Lean) that the agent then goes and impls.
If the impl does something I didn't want, I go back to improving the formal reqs. Otherwise, I get something that's provably what I asked for.
13
I learned that because 1) there is a limited supply of NIGHT and 2) Midnight block producers are paid by the treasury, their rewards will eventually reach 0. There's an expectation that governance should have a solution before that point.
29
@MacroAlf @MrBlonde_macro @prometheusmacro @LynAldenContact We trust the market's prediction the Fed will cut rates earlier than projected but don't trust the market's equities outlook since the impending catastrophic downturn isn't yet priced-in. How do we reconcile?
1
@JamesClear I've written down a number of checklists that represent stacked habits that I don't want to have to memorize. Do you do anything to tastefully display checklists in relevant locations? I'm trying to avoid having sticky notes all over my house.
Your wordle horoscope:
1/6: no skills, 'adieu' just hit
2/6: best day for years to come
3/6: good day
4/6: meh day
5/6: idiot but alive
6:6: this day hangeth by a thread
X/6: death lay ahead, do not leave bed
4
@hereandnow I missed the name of the person you were talking to about parametric insurance. Where would I find that info? Thanks!
I frack, you frack, we all frack for fracking. I can't believe Biden/Harris would ban the most relatable activity on Earth. Lost my vote.
3
biden:
"in fact"
"you at home"
"91 I mean 19 I mean 200 million I mean 200 thousand"
trump:
"chris"
"chris"
"you'd be surprised 😏"
"CHRIS"
2
Hey @thecodingtrain , the videos on creating perfectly looped gifs from 4d noise are awesome. Are there any techniques to do similar things when the particles within the sketch experience attraction and repulsion or exhibit any emergent behavior? gfycat.com/sorrowfuldependab…