Sameera Horawalavithana (@SamTube405) header
Sameera Horawalavithana (@SamTube405) profile picture

#NLP #Multimodal #AIScientists #AgentSecurity Scientist @PNNLab PhD @cseUSF Opinions here are my own and do not represent my employer. Proud 🇱🇰 Live 🇺🇸

Tampa, FL
Joined November 2010
Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)Photo by Sameera Horawalavithana (@SamTube405)
Sameera Horawalavithana retweeted
The next frontier is AI that improves itself, and we believe everyone should know how to build it. This semester, Ao @ao_qu18465 and I are teaching 'How to AI an AI for Almost Anything' @MIT @medialab -- a new course on self-evolving AI systems that recursively improve from experience, feedback, memory, and exploration over long interaction horizons. The key idea: intelligence shouldn't stop improving when training ends. We cover four levels of evolution: (1) context evolution through prompt optimization and context management, (2) harness evolution through memory, tools & skills, workflow optimization, and multi-agent systems, (3) model evolution through RL, distillation, test-time training, self-play, and continual learning, (4) task evolution through generating increasingly complex synthetic tasks, simulations, evaluations, and environments. Slides, readings, and videos available: mit-mi.github.io/how2ai-cour… We thank @Kimi_Moonshot and @PrimeIntellect for generously sponsoring API credits!
45
160
12
1,185
67,587
PhD internships opportunities to work with the language intelligence team @PNNLab to work on AI for science and legal/permitting domains. please apply. #Internship #NLP careers.pnnl.gov/jobs/12149?…
1
2
210
🧵 1/ New preprint drop: "Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models" 🦙 If you swap a better LLM backbone into your VLM, do you get a better VLM? Short answer: not really. Longer answer: it's more interesting than that.
1
1
114
4/ some VLM capabilities only emerge in the newest LLM generation. Other tasks dominated by visual understanding barely improve regardless of which LLAMA you plug in
1
62
Streamlining federal permitting with AI 📄🖥️⏩ PNNL researchers are using AI to bring valuable data distributed across hundreds of federal government agencies into a single dataset that's crucial for modernizing permitting technology for the 21st century.
4
14
1
36
6,631
Sameera Horawalavithana retweeted
CEQ coordinated with @ENERGY’s @PNNLab PermitAI project on the release of NEPATEC 2.0, a major accomplishment on the road to a simplified, speedier Federal permitting and environmental review process.
Streamlining federal permitting with AI 📄🖥️⏩ PNNL researchers are using AI to bring valuable data distributed across hundreds of federal government agencies into a single dataset that's crucial for modernizing permitting technology for the 21st century.
1
1
8
611
We are organizing AI4Permitting Workshop on April 29th at the NAEP 2025 conference in Charleston, South Carolina. Learn how to speed up National Environmental Policy Act (NEPA) processes at our upcoming workshop at NAEP 2025. pnnl.gov/events/using-nepa-c… #AI4Permitting #NEPA
86
Sameera Horawalavithana retweeted
We also currently have a kaggle competition open, which is running LLM evaluations on a QuAD benchmark specific to understanding permitting documents - deadline is june 30! kaggle.com/competitions/llm-…
1
2
9
1,234
Sameera Horawalavithana retweeted
This corpus is part of DOE's voltAIc initiative, which is using LLMs to accelerate permitting processes This corpus was developed in partnership with @PNNLab - you can find more details about their work on this project here: pnnl.gov/projects/policyai
1
2
1
13
811
Sameera Horawalavithana retweeted
🚨if you care about fixing NEPA environmental permitting OR are looking for new high-quality domain text corpuses to build LLMs on top of... DOE just released a 3.6B token corpus of federal permitting documents on huggingface! This corpus includes...
1
23
1
105
29,828
Sameera Horawalavithana retweeted
The video of the Lytle Lecture I gave at University of Washington last week is available. Title: "Objective Driven AI: Towards Machines that can Learn, Reason, and Plan" Lytle Lecture Page: ece.uw.edu/news-events/lytle… Slides: drive.google.com/file/d/1e6E… Video: youtu.be/d_bdU3LsLzE?si=UeLf…
26
250
14
1,171
575,132
📢 Accepted papers are up! 🔗: an-instructive-workshop.gith… Check out the range of cool new work on instruction training!
14
23
3,338
📢📢 I am recruiting PhD students for our group at Auburn to work on open-world event understanding and active embodied vision! If you are interested in working with our group, reach out! More info: saakur.github.io/group.html Please RT!
2
17
20
4,744
Sameera Horawalavithana retweeted
A new multimodal model LLaMA-SciTune for science-focused visual and language understanding. 📄 arxiv.org/abs/2307.01139v1
1
123
Sameera Horawalavithana retweeted
SCITUNE: Aligning Large Language Models with Scientific Multimodal Instructions Abs: arxiv.org/abs/2307.01139 Pdf: arxiv.org/pdf/2307.01139.pdf Presenting SciTune, a tuning framework to improve large language models' (LLMs) ability to follow scientific multimodal instructions. SciTune includes two stages: scientific concept alignment to learn across various scientific visual signals and textual signals, and scientific instruction tuning to fine-tune on a multimodal scientific reasoning task. LLaMA-SciTune, surpasses human performance on the ScienceQA multimodal reasoning benchmark and performs significantly better than SoTA vision-language models in a variety of scientific image understanding tasks with zero-demonstrations during the inference time.
2
2
147
Sameera Horawalavithana retweeted
Almost everyone I know working in AI these days feels one step away from total burnout. I took the time to take you behind the curtain and know what people on the state-of-the-art AI are struggling with: robotic.substack.com/p/behin…
12
132
42
684
338,118
Sameera Horawalavithana retweeted
Really nice survey on augmenting language models with reasoning skills and the ability to use tools. Includes prompting techniques, use cases, and applications. A must-read! arxiv.org/abs/2302.07842
9
52
4
233
31,629