Max Bartolo (@max_nlp) on Robust Real World Agents Many LLMs are great at static benchmarks but in reality they perform worse on real world inputs -- we need benchmarks that beat the AI, so we need adversarial ways to collect benchmarks -- such data will have a richer distribution of complex reasoning required compared to SQuAD style QA takes. -- DynaBench, HLE etc are some examples -- but it is more expensive -- Also applies to agentic LLMs -- usually collecting such data requires human feedback But human feedback is not gold standard. -- people tend to prefer generations that are formatted nicely rather than factual correctness -- for the same model, you can play with assertiveness through prompt. Perceived error rates by humans is higher for models when they are unassertive, but in reality, assertive responses contain more factual errors. Verifiable reward is powerful, to solve some of the problems with human feedback (but reward models trained on human feedback may also resurface these problems) Models fail even after closing the loop due to other reasons. Super interesting example on how many R's in blueberry. We fixed R's in strawberry but now blueberry has a problem. Other parts of the talk: is more tokens/context better, what kind of pretraining data helps reasoning.