Pinned Tweet
Should I use Macro F1 or Accuracy? Why not Kappa? Why do some use this, and others that? What's actually evaluated here? 😵💫
Happy to share the final version of this paper on multi-class classification evaluation:
direct.mit.edu/tacl/article/…
#machinelearning #nlproc #ml
Happy to share this preprint finally!
Embedding Models Measure in Peculiar Ways
arxiv.org/abs/2609.20821
This is a concerning trend.
Are people trying to "boost" their CV with a paper or is it AI startups trying to show "AI science"?
What's the deal with this? There's no material reward.
juri retweeted
TMLR has faced a deluge of submissions, necessitating stricter desk rejection policies due to limited reviewer capacity
Co-EiC Nihar Shah reached out to authors of 10 papers slated for desk reject. Could they answer questions about their *own* submission?
medium.com/@TmlrOrg/asking-a…
juri retweeted
I don't get it. When it comes to code, AI should be way smarter than me. Yet there are dozens of examples where I ask the LLM to optimize something, it completely fails, I suggest "What if we do this?" and it says "Wow, great idea. That will work." And of course, I'm skeptical--maybe it's being sycophantic--but then we implement my idea and it's legitimately much more performant!
I am not some hotshot software engineer; I don't understand how I'm able to easily find improvements that frontier LLMs overlook. And this happens all the time!
What am I bringing to the table that it doesn't already have?
hey, what's with those reviewers who give
excitement 4 (this paper is exciting)
soundness 4 (this paper is strong)
overall score 2.5 (maybe reject)
i never understand those reviewers, like, how do they think?
juri retweeted
Does your company need more attention or investment? Just tell the world that there's a 10% chance your product will kill us all.
juri retweeted
Years ago you only needed papers when applying for job as post-doc or faculty. Now secondary-school students are using LLMs to write papers, to help them get into a good UG university... Inevitable result is wave of AI slop, which (as below) "authors" may not even understand.
We are seeing a new trend in submissions to @arxiv (and presumably to conferences and journals): Authors submitting papers whose contents they likely do not understand. 1/
juri retweeted
Anthropic and OpenAI delaying their IPO, because their S-1 will reveal that they are bleeding money, and have no path to profitability.
Solution? "AI slowdown", so they cut costs for training new models.
It has everything to do with IPO, and nothing to do with safety.
juri retweeted
I'm fascinating by how miserly AI progress is.
A lot of us assumed that once AI was as capable as it is now, we'd see insane progress in the things we actually cared about like number of useful projects completed. But that hasn't really happened.
It amazes me that the machines have literally proved a millennium prize problem, one of the hardest challenges in math, and yet somehow the result is still almost useless to us.
It's just a mountain of crap we humans have to dig through in hopes of salvaging crumbs of insights.
AI gives answers but in the least useful way possible.
Received three reviews. The first is short, helpful, and positive, human written. The other two are long lists with super picky minor details, no major issue outlined, and very negative. Is this situation normal now in the academic review system?
Evaluating Embeddings Dynamically in 200+ Languages with Automatically Induced Minimal Pairs.
Just accepted at EMNLP ("main").
Fine work by Andrianos Michail and others.
#EMNLP2026 #nlproc
arxiv.org/abs/2607.00171
juri retweeted
LLM review weirdness...
Just renaming an uploaded pdf from "paper.pdf" to "paper_final_draft_pdf_ready_for_review.pdf" boosts average scores
(gpt-5.6-terra)
i think that exploding paper submissions could also be mitigated somewhat by stopping the conditioning of phd completion on "n papers at top conferences." there need to be better ways. maybe back to a writing a coherent monograph? or accepting *good* papers regardless of venue?
Then why not create an AI-first conference? No cap on number of submissions, full AI reviewing and decisions, papers can be written up to 100% w/ AI.
This quoted post is unavailable.
juri retweeted
Brief update on the progress of the EMNLP 2026 AI Reviewing Experiment and a timeline for releasing the AI reviews to opted-in authors 2026.emnlp.org/ai-reviewing-…
#EMNLP2026
Very challenging! i dropped myself out because i couldn't come up with more than one example from German<->English that are not solved by Sota LLMs. Be smarter than me and join the effort!
juri retweeted
When Claude writes code, it signs the commit: Co-Authored-By: Claude. I'm paying for the model, and it still takes a credit line. Okay, fine.
But co-authorship is a claim about contribution. And that logic should work both ways.
So what about all those people who involuntarily "contributed" the training data? You know, half the internet. Don't they deserve co-authorship on the models?
