
Research Scientist at Meta Superintelligence Labs | ex-@BellLabs
London, England
Joined August 2017
- Tweets178
- Following144
- Followers501
- Likes409
Akhil Mathur retweeted
📣 Our recent work at Meta: Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
Rubric-based judging and reward modeling are powerful for open-ended tasks, but scaling requires controlled generation, not just free-form lists from an LLM.
Proud of this work from @LisaAlazraki's internship at Meta Superintelligence Labs. We take a systems-level view of agentic AI and show that capabilities emerge from coordination between agents, not just parametric scaling. #agents #msl #meta
It’s here! Meet Llama 3, our latest generation of models that is setting a new standard for state-of-the art performance and efficiency for openly available LLMs.
Key highlights
• 8B and 70B parameter openly available pre-trained and fine-tuned models.
• Trained on more than 15T tokens, 7x+ larger than Llama 2's dataset!
• Improved tokenizer with vocabulary of 128K tokens for better performance.
• State-of-the-art performance across industry benchmarks.
• New capabilities, including enhanced reasoning and coding.
• 3x more efficient training than Llama 2.
• New trust and safety tools with Llama Guard 2, Code Shield, and CyberSec Eval 2.
• Integrated into Meta AI, and available in more countries across our apps.
• And, just the beginning with more models and new capabilities coming soon!
Visit the Llama 3 website to read more and download the models. llama.meta.com/llama3
6
494
Akhil Mathur retweeted
1/5 Tired of self-supervised models that can't adapt to new data?
In our #WACV2024 paper, we propose Kaizen - an end-to-end approach for continual learning that performs knowledge distillation across both the pre-training and fine-tuning steps.
👉 arxiv.org/abs/2303.17235
Akhil Mathur retweeted
🤿 What is latent masking and why is it relevant to multimodal learning?
In our paper in ML4MHD at #ICML2023 we presented CroSSL, a new model that masks intermediate embeddings to improve multimodal learning.
📖 Paper: arxiv.org/abs/2307.16847
🎥 Video: drive.google.com/file/d/1lF5…
Akhil Mathur retweeted
This is huge: Llama-v2 is open source, with a license that authorizes commercial use!
This is going to change the landscape of the LLM market.
Llama-v2 is available on Microsoft Azure and will be available on AWS, Hugging Face and other providers
Pretrained and fine-tuned models are available with 7B, 13B and 70B parameters.
Llama-2 website: ai.meta.com/llama/
Llama-2 paper: ai.meta.com/research/publica…
A number of personalities from industry and academia have endorsed our open source approach: about.fb.com/news/2023/07/ll…
So happy to see this important work published. Model fairness should be a key consideration when we optimize machine learning pipelines for edge devices. Our paper offers a framework to approach on-device ML fairness, along with extensive experimental findings for speech models.
You’re unable to view this Post because this account owner limits who can view their Posts. Learn more
6
689
Great progress towards efficient and adaptive FL on ultra-constrained devices. Work led by @VincentMo6 during his internship at @BellLabs.
Federated Learning for constrained edge devices need to become more efficient and adaptive. In this preprint, we offer Centaur: a framework that adaptively partitions FL training across multiple clients owned by each user.
PDF: arxiv.org/abs/2211.04175
1/4
Akhil Mathur retweeted
🥁🥁 Join us for ACM SIGCHI Symposium for 𝐇𝐂𝐈 𝐚𝐧𝐝 𝐅𝐫𝐢𝐞𝐧𝐝𝐬 at IIT Bombay, India, 9-11 December 2022!🧵
Speakers: namastehci.in/#speakers
November 9th: Applications due
namastehci.in/#registration
🙏
Anupriya, @akhilmathurs, Anirudha Joshi, @Pushpendra_S__, @nehakumar
All train services between London and Cambridge are delayed because there is a swan 🦢 sitting on the track 🤷🏻♂️🤦♂️
1
15
Charlie Dean was outside the crease for more than 85% of all balls she started at the non-striker's end.
"Spirit of cricket!" 🙄
Went back to the full match replay. Charlie Dean was leaving her crease early starting with her 2nd ball at the non-striker's end in the 18th over. Ball still in bowler's hand. Dean is never looking at the bowler to see if/when the ball has been released. Basic lack of awareness.
5
Shohreh Deldari (@ShohrehDeldari) from RMIT is now presenting her super cool work on cross modal self supervised learning. #UbiComp2022
Hyunsung Cho (@hciresearcher) is presenting her work FLAME 🔥 which shows how to make federated learning work in multi-device environments #UbiComp2022
Brilliant talk today by @IanTangC on our collaborative self-supervised work aimed at making HAR training data-efficient. #UbiComp2022 @ubicomp
In our upcoming paper at #IMWUT @ubicomp, we present ColloSSL: a technique for collaborative self-supervised learning among a group of devices. 🧵 (1/n) arxiv.org/abs/2202.00758
"At the stroke of the midnight hour, when the world sleeps, India will awake to life and freedom.
Happy 75th Independence Day! The journey has just begun #IndiaAt75
Looking forward to presenting our work and learning more about ML efficiency @ Deep Learning Indaba. Kudos to the incredible organizing team for putting this event together.
Replying to @sarahookr
The exciting part is the incredible in-person talks and lineups we have planned in Tunis. Including:
- @KateKallot, Head of emerging areas at NVIDIA
- @akhilmathurs, Principal Research Scientist at Bell Labs
- @zngu, Associate Professor, University of Edinburgh
If you are attending #ICML2022, please join our spotlight talk on self-supervised FL on July 19 and poster presentation on July 21. Please feel free to DM if you are up for a chat on FL, self-supervision, and embedded ML.
icml.cc/virtual/2022/spotlig…
Glad to share that our work FLAME has been accepted to IMWUT '22. We explore algorithmic & system challenges for federated learning in the presence of multiple data-generating devices on a user. Proud of our intern @hciresearcher who led this work. arxiv.org/abs/2202.08922 @raswak
An excellent summary of our upcoming ICML paper on unsupervised FL. The paper is out on arXiv arxiv.org/abs/2205.11506. Also, a big thank you to the Flower framework flower.dev/ which helped us scale our FL experiments.
Hello, world! We present to you 🎶Orchestra🎶, an unsupervised framework for federated learning! The paper was recently accepted at #ICML.
Abs: arxiv.org/abs/2205.11506




























