Cohesive Systems retweeted
Interesting paper: coding agents do well on tasks they can script their way through (grep, AST rewrites). Accuracy falls when they have to reason about behavior, like identifying which lines in two nearly identical classes actually behave differently.
On SWE-bench, the number of reading and analysis steps correlated more strongly with failure than lines edited.
Finding code isn’t the same as understanding its behavior. Better comprehension tools will help…but why not make system meaning explicit so agents don’t have to reconstruct it from implementations?
arxiv.org/abs/2610.10610
Make the model the implementation!
When an LLM writes a formal spec from requirements plus possibly buggy code, it can produce a spec that the buggy code satisfies: either too weak, or shaped by the implementation. The verifier passes it. Advanced prompting didn't fully fix it.
This is verification vs. validation. Proving code meets a spec says nothing about whether the spec captures intent.
One way out: make the model the implementation. Compile it instead of hand-writing code that's supposed to match it. That closes the model-to-code gap. Validation remains, but it happens against a model a domain expert can read and explore through generated scenarios, not against thousands of lines of code.
arxiv.org/abs/2610.32053
Agents make code cheap. Understanding the system is still the bottleneck.
Cohesive is a language for describing systems: their rules, workflows, and guarantees.
People and agents work from the same model. Machines check and compile it.
Here’s a brief introduction.
Cohesive Systems retweeted
The AI era has given formal methods a reemerging role.
As code production becomes increasingly automated, formal methods are moving upstream, from verifying code to formalizing intent. The scarce resource is no longer writing software; it’s expressing precisely what the software should do.
That makes executable specifications, constraints, and semantic models far more valuable than they were when humans wrote every line of code.
Why people don't use formal methods hillelwayne.com/post/why-don…



