Akshay Kulkarni

Are Firewalls All You Need, or Stronger Benchmarks? Benchmarking is critical for understanding and comparing the security of tool-calling agents. As attacks evolve and defenses adapt, researchers need consistent, realistic, and reproducible evaluation frameworks to identify true progress and avoid misleading conclusions. Several recent benchmarks, such as AgentDojo, Agent Security Bench, and InjecAgent, aim to simulate real-world attack scenarios. However, our analysis further reveals that many of these benchmarks do not model real-world situations appropriately and sometimes employ skewed metrics to gauge performance. In such cases, even weak defenses may seem deceptively effective. We highlight these limitations and fix them through our proposed standardized benchmarking best-practices. Source: arxiv.org/pdf/2510.05244 @rishika2110, @KevinKasa98, @AbhayPuri98, @GabrielHuang9, @irinarish, Graham W. Taylor, @DjDvij, @alex_lacoste_ - @ServiceNowRSRCH, @Mila_Quebec, @UMontreal, @VectorInst, @uofg #PromptInjection #LLMSecurity #AIAgents #AgentSecurity #AIEvaluation #Guardrails #CyberSecurity #AISafety #MilaQuebec #ServiceNowResearch #VectorInstitute #UniversityOfGuelph