Roy Lee

✏️LongWriter-Zero accepted to ICLR as an oral! We found that after SFT, RL couldn’t reach long enough outputs—length reward stayed low. So we skipped SFT and did zero RL from the base model. 🔍Mid-training with the right data, and robust reward modeling, turned out to be key.

Happy to share that the @GoogleDeepMind Gemini team is starting a new research team in Singapore! This new team will be focused on advanced reasoning, LLM/RL and improving bleeding edge SOTA models such as Gemini, Gemini Deep Think and beyond. 🔥 This team will be led by yours truly and reports up to Quoc Le (@quocleix)'s broader team in Mountain View which was recently in the center of both IMO gold medal and ICPC gold medal breakthroughs with Gemini Deep Think, amongst many other significant Gemini advancements. 🚀 We’re starting out with a very small but intensely capable force because talent density is key over anything else in the LLM era. Over the past few months, we have gone around and gathered the best of the best talent (in the region and beyond) and I’m confident we’ll have a super cracked team very soon. If you are interested in joining and have made truly exceptional contributions in any domain or area, (engineering and/or research etc) please contact me. This is quite an exciting time, with the Gemini / GenAI team at Google Deepmind leading the charge at the frontier. This is also the best opportunity to be on the critical path to AGI from the sunny island of Singapore. 🏝️ Many thanks to leadership support from @quocleix @JeffDean @benoitschilling, @EugenieRives and @demishassabis for the support of this team. Wonderful and fun image generated by Nano Banana 👇