Soheil Bahrampour (@soheilbp) profile picture

Machine Learning/ Engineering Manager @ Google, prev: Deep Learning @ Bosch Research, PSU Machine Learning PhD

San Francisco Bay Area, CA
Joined May 2012
Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)Photo by Soheil Bahrampour (@soheilbp)
Soheil Bahrampour retweeted
In era of pretraining, what mattered was internet text. You'd primarily want a large, diverse, high quality collection of internet documents to learn from. In era of supervised finetuning, it was conversations. Contract workers are hired to create answers for questions, a bit like what you'd see on Stack Overflow / Quora, or etc., but geared towards LLM use cases. Neither of the two above are going away (imo), but in this era of reinforcement learning, it is now environments. Unlike the above, they give the LLM an opportunity to actually interact - take actions, see outcomes, etc. This means you can hope to do a lot better than statistical expert imitation. And they can be used both for model training and evaluation. But just like before, the core problem now is needing a large, diverse, high quality set of environments, as exercises for the LLM to practice against. In some ways, I'm reminded of OpenAI's very first project (gym), which was exactly a framework hoping to build a large collection of environments in the same schema, but this was way before LLMs. So the environments were simple academic control tasks of the time, like cartpole, ATARI, etc. The @PrimeIntellect environments hub (and the `verifiers` repo on GitHub) builds the modernized version specifically targeting LLMs, and it's a great effort/idea. I pitched that someone build something like it earlier this year: teapaw.org/karpathy/status/188467… Environments have the property that once the skeleton of the framework is in place, in principle the community / industry can parallelize across many different domains, which is exciting. Final thought - personally and long-term, I am bullish on environments and agentic interactions but I am bearish on reinforcement learning specifically. I think that reward functions are super sus, and I think humans don't use RL to learn (maybe they do for some motor tasks etc, but not intellectual problem solving tasks). Humans use different learning paradigms that are significantly more powerful and sample efficient and that haven't been properly invented and scaled yet, though early sketches and ideas exist (as just one example, the idea of "system prompt learning", moving the update to tokens/contexts not weights and optionally distilling to weights as a separate process a bit like sleep does).
Introducing the Environments Hub RL environments are the key bottleneck to the next wave of AI progress, but big labs are locking them down We built a community platform for crowdsourcing open environments, so anyone can contribute to open-source AGI
247
842
171
7,241
953,771
Soheil Bahrampour retweeted
ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark abs: arxiv.org/abs/2303.13648
12
145
23
698
167,244
Soheil Bahrampour retweeted
What actually happens when you call .backwards() in @PyTorch? Autograd goodness 🪄! PyTorch keeps track of all of the computations you’ve done on each of your tensors and .backwards() triggers it to compute the gradients and stores them in .grad. 1/3
4
89
3
448
Soheil Bahrampour retweeted
Since our last update, Google has accelerated our work to help people stay safe, informed & connected. Here’s the latest in our ongoing global #COVID19 response. We’ll keep doing everything we can to help people take care of themselves & their communities blog.google/inside-google/co…
91
428
54
1,649
Soheil Bahrampour retweeted
Video of my moment of glory at @icmlconf 2019 tinyurl.com/yy5njrn7 and the slides tinyurl.com/y3chg5lv. A great thanks to @kamalikac and @rsalakhu (and many other people) for the time and efforts they devoted to the community when organizing such a large event.
10
37
1
224
Soheil Bahrampour retweeted
We’re excited to share that weight pruning is now part of the TensorFlow Model Optimization Toolkit! Learn how sparsity can dramatically reduce model sizes with negligible accuracy loss. Read here → goo.gle/2vYWBrO
7
224
27
659
Soheil Bahrampour retweeted
Videos of all sessions at ICLR 2019 (including live stream) facebook.com/pg/iclr.cc/vide… facebook.com/yann.lecun/post…
58
5
144
Soheil Bahrampour retweeted
Our research group is starting a (technical) blog! First post, by @RICEric22, covers provable adversarial defenses locuslab.github.io/2019-03-1…. Each post is downloadable as a Jupyter notebook, so you can recreate all the examples. More info about blog here locuslab.github.io/2019-02-2….
35
3
152
Soheil Bahrampour retweeted
On behalf of the whole Google Research & @GoogleAI community, I was excited to put together a post describing some of the work that we collectively did in 2018. I hope you enjoy it! Thanks to everyone who helped make this work possible! ai.googleblog.com/2019/01/lo…
25
582
67
1,942
Soheil Bahrampour retweeted
Announcing the launch of Dataset Search, a new way for researchers to find the datasets they need, wherever they’re hosted, whether it’s a publisher's site, a digital library, or an author's personal web page. Learn more at goo.gl/BYSouA
51
2,659
433
4,508
Soheil Bahrampour retweeted
There is a Roadster in space. And it has a live feed: youtube.com/watch?v=aBr2kKAH… definitely the fastest car :)
15
205
22
799
Take advantage of distributed training on your own cluster by converting a Keras model to an Estimator in @TensorFlow 1.4 → cloud.google.com/blog/big-da…
43
91
Soheil Bahrampour retweeted
Facebook streaming videos of our dual tracks are available from the NIPS schedule and from nips.cc/Conferences/2017/Vid…
3
126
7
279
Soheil Bahrampour retweeted
Ever wonder what "dropout regularization" really means? 🤔 This and ~200 other #ML definitions at your fingertips in this ML Glossary 📖 Check it out here ↓goo.gl/zqqjKj
3
161
9
349
Soheil Bahrampour retweeted
GANs for dataset augmentation received a best paper award at #CVPR17 ! cvpr2017.thecvf.com/program/…
3
127
8
414
Soheil Bahrampour retweeted
A beautiful mind.
33
849
70
3,322
Soheil Bahrampour retweeted
Rest In Peace Maryam. You were an inspiration to mathematicians the world over. tehrantimes.com/news/415087/…
23
276
16
620