@GenBench

State-of-the-art generalisation testing in NLP. Tag us for a RT of your NLP generalisation paper tweet!

genbench.org/
Joined April 2022
  • Tweets193
  • Following15
  • Followers430
72Media
The GenBench workshop is back! Do you work on generalisation (benchmarking) in #NLProc? Submit to the 2nd edition (genbench.org/workshop/) co-located with #EMNLP2024. We have a regular track and a ✨collaborative benchmarking task (CBT)✨ that's fully LLM-focused this year (1/6)

GenBench Workshop 2024

The second workshop on generalisation (benchmarking) in NLP

genbench.org
1
12
5
22
12,605
That's a wrap! We (@glnmario, @christos_c, @_dieuwke_, @vernadankers, @khuyagbaatar_b, @a_kazemnejad & @ryandcotterell) thank all presenters, authors, reviewers and attendees!! The keynotes, the cats 😻, the posters, the talks and the lively panel: it was fantastic👏 🔥
7
45
2,883
so proud of @HayleyRossLing for getting a best paper award at @GenBench this year!! 🎉🪅🎉 I'm sure @TeaAnd_OrCoffee would be too :) check out our paper and share if you think homemade cats are cats!
Hayley Ross@HayleyRossLing24 Oct 2024
New paper with @najoungkim and @TeaAnd_OrCoffee testing if LLMs can draw adjective-noun inferences like humans! Turns out they often can, and even generalize to unseen combinations. But they're more optimistic about "artificial intelligence" than humans. arxiv.org/abs/2410.17482
1
6
1
60
3,496
Woohoo go tinlab! Congrats @HayleyRossLing @TeaAnd_OrCoffee @najoungkim!!
GenBench@GenBench16 Nov 2024
Replying to @GenBench
Best paper!
2
16
1,273
Congratulations!
Najoung Kim 🫠@najoungkim16 Nov 2024
so proud of @HayleyRossLing for getting a best paper award at @GenBench this year!! 🎉🪅🎉 I'm sure @TeaAnd_OrCoffee would be too :) check out our paper and share if you think homemade cats are cats!
3
238
Closing remarks and best paper award by @vernadankers
1
2
12
907
Best paper!
2
1
7
1,409
Congrats to all the authors!
2
92
And we also have an honourable mention!
1
103
Come listen to the hot takes of our panelist in the Brickell room! Do we still need generalisation evaluation? 🧐 #GenBench2024 #EMNLP2024
4
15
1,492
Still at the poster session? Come join us for keynote 3 by @sameer_!
1
5
741
Did you miss the GenBench poster session? Don't worry we've got you, here are (nearly all) posters! 😉 #GenBench2024 #EMNLP2024 Next up: keynote by Sameer Singh at 3!
2
13
831
Spotlight time! Mirella Bueno on MLissard: Multilingual Long and Simple Sequential Reasoning Benchmarks aclanthology.org/2024.genben…
1
1
3
537
more replies
Continuing with Bastian Bunzeck, presenting The SlayQA benchmark of social reasoning: testing gender-inclusive generalization with neopronouns aclanthology.org/2024.genben…
1
1
3
87
Last spotlight presentation: MMLU-SR: A Benchmark for Stress-Testing Reasoning Capability of Large Language Models aclanthology.org/2024.genben… Unfortunately the authors couldn't make it, the work is kindly presented by their colleague Hengyi Wang 🙏
1
1
71
Join us for our second keynote by Olmo co-lead @kylelostat
1
4
16
1,225
He got all the room snickering already at slide 3! 😁
1
2
97
Plus more cat pictures! 😻😻
1
93
Oral presentation two with @sagnikrayc Investigating the Generalizability of Pretrained Language Models across Multiple Dimensions: A Case Study of NLI and MRC aclanthology.org/2024.genben…
1
3
7
918
2
2
217
Load more