@devjeetrr

PhD student at Washington State University

github.com/devjeetr
Joined July 2013
  • Tweets38
  • Following433
  • Followers50
Devjeet Roy retweeted
Sarah Fakhoury@fakhourysmMay 7
1/ In the AI era, specifications are becoming the primary interaction mechanism between humans and coding agents. Inferring, validating, and maintaining them is foundational to trustworthy software engineering.
1
1
1
2
289
Devjeet Roy retweeted
Gabe Orlanski
@GOrlanskiMay 4
Measuring learning, not just capability, is fundamental to evaluating continual learning methods. Continual Learning Bench is built around exactly that! Was a pleasure to contribute to this, check out the full write up:
Parth Asawa
@pgasawaMay 4
Today, we’re releasing Continual Learning Bench 1.0: the first, realistic benchmark for measuring how AI systems can improve in online settings. Benchmarks today assume models are stateless. Each example is independent, and once a system finishes a task, it moves on as if nothing happened. But deployed AI systems should learn from experience. We tested 10+ frontier systems against novel, expert-validated tasks and find there’s still plenty of headroom for learning. (1/n)
3
21
1,745
Devjeet Roy retweeted
Gabe Orlanski
@GOrlanskiApr 24
Very excited to announce the v1.0 of SlopCodeBench release: - Doubling the size of the dataset - @harborframework support - scb-check: a CLI that flags slop anti-patterns - Way more model results scbench.ai github.com/SprocketLab/slop-… 🧵
2
11
8
78
34,772
Devjeet Roy retweeted
Snorkel AI
@SnorkelAIApr 23
Our MLSys 2026 paper is live on arXiv: “Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes.” @realjustinbauer @Walshe_tech @pham_derek @harit_v @ArminPCM @fredsala and @paroma_varma present a comprehensive empirical study of open-source SLMs after RLVR in low-data regimes, revealing that dataset composition matters more than dataset size for scaling performance across number counting, graph, and spatial reasoning tasks. Read the paper: arxiv.org/abs/2604.18381
8
1
46
527,389
Our #MLSys2026 paper is live on arXiv 📄 We ran a systematic study of RLVR in low-data regimes across 3 procedurally generated benchmarks (counting, graph, spatial reasoning). Key finding: dataset composition matters more than dataset size. arxiv.org/abs/2604.18381

Learning from Less: Measuring the Effectiveness of RLVR in Low...

Fine-tuning Large Language Models (LLMs) typically relies on large quantities of high-quality annotated data, or questions with well-defined ground truth answers in the case of Reinforcement...

arxiv.org
2
10
33
2,243
Devjeet Roy retweeted
Gabe Orlanski
@GOrlanskiMar 27
We found that agents generate progressively worse code with each iteration. Real developers do not. SlopCodeBench is the only eval that faithfully measures quality degradation on iterative, long-horizon coding tasks. arxiv.org/abs/2603.24755 scbench.ai 🧵
44
100
52
720
184,604
Devjeet Roy retweeted
Alex Gu@minimario172918 Dec 2025
AI coding assistants generate a lot of slop. Now there's a way to measure it: SlopCodeBench!
Gabe Orlanski
@GOrlanski18 Dec 2025
Often, a bug I have had to fix has been tied directly to overly verbose or defensive code from an AI agent that has eroded my project as the specs changed. Why isn't there an eval for this? There is now. Very excited to announce SlopCodeBench scbench.ai
3
24
4,292
If you want to add your agent to the harness/run the evals/or anything related, please reach out. I would be more than glad to help 😀
1
6
247
Often, a bug I have had to fix has been tied directly to overly verbose or defensive code from an AI agent that has eroded my project as the specs changed. Why isn't there an eval for this? There is now. Very excited to announce SlopCodeBench scbench.ai
1
17
9
52
15,464
What's the future of AI for Software Engineering? 🤖 Join Alex Gu (@minimario1729) (MIT; StarCoder, CRUXEval, LeanDojo contributor) tomorrow at the #DL4C workshop! He'll cover current challenges in AI for SE and promising directions for what lies ahead. #ICLR2025 #ICLR
1
4
1
13
5,085
We're excited to welcome our fourth speaker @taoyds, Assistant Prof at @HKUniversity and director of @XLangNLP. His groundbreaking work in grounding language into code and actions spans digital and physical environments, aiming to democratize data science and enhance human-computer interaction. Join us at the 3rd Deep Learning for Code workshop at #ICLR2025, this Monday in Singapore! 🇸🇬 #DL4Code #NLP #AI
1
4
212
Go beyond code completion! Baptiste Rozière (@b_roziere) presents his work at @MistralAI automating code tasks (IDE tools, agents) for better developer focus. Catch him this Sunday at #DL4C! #ICLR2025 #iclr
2
7
414
Just 6 days until #DL4C! 🗓️ Daniel Fried (CMU / Meta AI) @dan_fried @AIatMeta will be sharing insights on how inducing functions from code makes LLM agents smarter and more efficient. Don't miss it! See you Sunday! #ICLR2025 #iclr
3
1
11
3,256
🚀 ICLR week is upon us! Join us at the #DL4C Workshop to hear Xingyao Wang (@xingyaow_) discuss LLMs evolving into SE agents, covering the CodeAct framework (code exec as action), the OpenHands platform (dev-like generalist agents), & SWE-Gym (real-world task training). @iclr_conf
2
1
13
4,691
Lots of great papers in – thank you all! Review will start from tmrw. Last chance to sign up to be a reviewer for #DL4C forms.gle/gPosJujrdAQqUvJG8 @iclr_conf #iclr #iclr2025 #iclr25

Program Committee Membership for Deep Learning for Code (DL4C) Workshop at ICLR 2025

We are planning to host the Deep Learning for Code (DL4C) workshop again this year! Currently, we are submitting the proposal and assembling the Program Committee. Claim your spot by filling out the...

docs.google.com
30
1
12
1,989
🚀Excited to share the 3rd Deep Learning for Code workshop is back at @iclr_conf'25! This year we’ll focus on emergent challenges in the field, e.g., agents, post-training, developer productivity, open science, and benchmarking for code Submit by Feb 3 dl4c.github.io🧵⬇️

The Deep Learning for Code Workshop | Deep Learning For Code

The Deep Learning for Code (DL4C) workshop brings together researchers from Machine Learning, NLP, Human-Computer Interaction, and Software Engineering to advance how AI systems write, understand,...

dl4c.github.io
1
10
4
18
23,914
What does it mean for the LLMs of Code to be open and responsible? Our fourth invited talk by Harm de Vries (@harm_devries) from @ServiceNowRSRCH and Leandro von Werra(@lvwerra) from @huggingface is about BigCode (bigcode-project.org/)! #ICML2023

Open and responsible development and use of LLMs for code

BigCode is an open scientific collaboration working on the responsible development and use of large language models for code

bigcode-project.org
1
5
14
1,222
Thank you to all authors who submitted their works to the DL4C workshop @ ICLR 2023! Instructions and Upload Form for Authors of Accepted Papers: forms.gle/LiTdK393NTz4VJwM8 Deadline: May 1st
4
5
696
📢We are excited to announce our incredible lineup of speakers for the Second DL4C Workshop @ ICLR 2023! Link: dl4c.github.io/speakers/ The speakers are:
1
15
2
71
17,374
📢 The deadline to submit to the DL4C Workshop is less than a month away! Link: openreview.net/group?id=ICLR… Submit by February 3rd

ICLR 2023 Workshop DL4C

Welcome to the OpenReview homepage for ICLR 2023 Workshop DL4C

openreview.net
1
8
2
21
6,019
Load more