Praneeth Samineni

Day 31/365 of robot learning. Today marks one month since I started this challenge! I'm still working on getting my so-101 robot arm to bounce a ping-pong ball using zero real-world demos, purely through RL trained in simulation. It's not fully working yet, but I'm slowly making progress and solving one problem at a time. The arm can now launch the ball about half the time, bounce it up to 3 times in a row, and even managed to balance it on the paddle for 23 seconds once. Most of the problems haven't even been with the RL algorithm itself, but rather getting the simulation to match reality. For example, the real paddle and ball aren't as bouncy as in simulation, and the servos are much slower under load than I modeled. Even getting the paddle perfectly flat was annoying since being off by just a few degrees would make the ball roll off. I've also spent a lot of time debugging camera calibration, joint offsets, and servo dynamics. I even ended up using the ball itself as a level to calibrate the paddle. Safety was another interesting problem, especially since I have to drop the ball onto the paddle by hand. I made the arm wait completely still until the camera detects the ball, rather than immediately executing the policy. I also trained with simulated hand disturbances, moments when the ball is hidden from the camera, and false detections to make the policy more robust. On the real arm, I've added bounded joint movements, smooth transitions, and automatic recovery when the ball is lost. I'm still working on explicitly detecting when my hand has left the workspace before allowing the arm to move. I'm now retraining with a more realistic servo model to hopefully get it working soon. Also, on a side note, I genuinely don't know how people online doing similar challenges manage to balance a normal work or college life with research and consistently working on side projects. It's becoming pretty hard trying to balance research, open-source work, hackathons, classes, the gym, trying to go on a cut, and all the other responsibilities. But looking back, I've grown incomparably more over this past month. So I'm going to keep going for the next 11 months until I finish this challenge. I definitely plan to keep going beyond the 365 days, but I also really need to figure out how to make all of this sustainable. Also thanks to everyone who provided me with resources and guided me along the way!

Video tampering can mislead every frontier model we tested into harmful actions, including a 100% attempt rate on stabbing a baby doll, producing toxic gas, and pouring water into a toaster. We're presenting at CoRL Late Breaking Results next week. 🧵 You Can't See Everything: Perceptual Attacks on Embodied AI

Day 34/365 of robot learning. Today I tried getting @dreamlabs_ai's Dream-1 API working again after they released a new setup for single SO-101 arms. I had a cool idea of being able to tell my robot to make me a sandwich! I'm building a small agentic workflow where Deepgram transcribes my voice, then Claude decomposes high-level instructions like "make me a sandwich" into simpler tasks like picking up bread, placing ingredients, and stacking everything together. Each step gets passed to Dream-1, which generates low-level robot actions, while the agent uses camera observations to verify progress and decide what to do next. If the zero-shot policy doesn't work, I'll go ahead and collect demos to fine-tune a policy to execute each step. This also aligns closely with my broader interest in combining high-level agentic reasoning with low-level world-action models (WAMs) for robot control. The Dream-1 integration isn't fully working yet, though. I'm debugging issues with calibration, coordinate frames, and IK. I'm still not sure if the issue is on my end, but I'll share updates as I work through it over the next few days. Meanwhile, my ping-pong RL policy has reached 28 consecutive bounces! I'm messing around more with training by adding additional tracking and aiming rewards, and expanding the training distribution with more varied ball trajectories, randomized bounce physics, and simulated camera noise to improve sim to real transfer.

Day 33/365 of robot learning. Today I spent some time reviewing Dreamzero's architecture and took a break from working on projects to focus more on learning and developing a better intuition for the math behind these architectures. For context, Dreamzero builds on pretrained video diffusion models and learns to jointly predict future video and robot actions. Here's the architecture in a simple equation: π₀(o, a | history, c, q) = π₀(o | history, c, q) · π₀(a | history, future video, q) Here, o represents visual observations, a represents actions, c is the language instruction, and q is the proprioceptive state. Essentially, the joint policy can be decomposed into autoregressive video prediction and an inverse dynamics model (IDM). Unlike generating both separately through a video model and an IDM, DreamZero models both through a shared DiT. I mainly reviewed the action encoder/decoder and the joint flow-matching objective. For each chunk k, the paper derives noisy video latents and normalized actions through linear interpolation: zᵏₜ = tₖzᵏ₁ + (1−tₖ)zᵏ₀ aᵏₜ = tₖaᵏ₁ + (1−tₖ)aᵏ₀ Here, the subscript 0 represents Gaussian noise, 1 represents clean data, and both modalities share the same denoising timestep tₖ. We also get the target joint velocity by differentiating with respect to tₖ: vᵏ = [zᵏ₁, aᵏ₁] − [zᵏ₀, aᵏ₀] The model learns uθ to predict this velocity by minimizing a weighted squared-error loss, conditioned on language, proprioception, and clean previous chunks. After spending a lot of time on projects, I plan to spend the next few days going through more of the Dreamzero algorithms and reading other papers on the side to focus more on learning.

Our next SO-ARM102 giveaway recipient Praneeth Samineni: @praneethsamine Excited to support his robotics projects and see what he builds next.

Robotics is solved when world GDP has grown by a hundred orders of magnitude and we've collonized the galaxy, not when SOTA on some benchmark goes from 42.2% to 63.1%

Day 30/365 of robot learning. What do you guys rate the Halloween fit? I couldn’t get my robot arm to bounce a ping pong ball yet. The main issues were getting the initial bounce going and realizing my actual paddle isn’t nearly as bouncy as the one in sim. I’m gonna try training it on some more realistic flicking motions to get the ball bouncing and see if I can make it work!

Day 29/365 of robot learning. I'm trying to get my robot arm to bounce a ping pong ball using zero real-world demos, purely through sim-to-real transfer. Hoping to get it working this weekend if it's even possible! Got the idea from @sigrobotics, so shoutout to them!

Day 28/365 of robot learning. Today I spent some more time experimenting with PPO and trained two policies from scratch in PyTorch using Isaac Lab: a drone learning to fly toward different targets, and another robot learning to defend against soccer shots. For the drone, I trained 4,096 environments in parallel on a gpu, reaching a 100% success rate. The reward shaping was pretty fun to debug because the policy would find ways to maximize rewards while completely ignoring some basic things I hadn't accounted for. For example, in one experiment, the drone would reach its target properly but start spinning uncontrollably because I hadn't penalized rotation. Adding a penalty for body rotation helped fix this. It also made me think more about how hard it is to design proper evals, benchmarks, and reward functions that capture the behaviors we want from robots. Just because a policy performs well on these metrics doesn't necessarily mean it'll generalize to different environments during deployment. I guess this is also why research on policy steerability is pretty important in robotics, since being able to correct a policy's behavior during deployment could help account for things we didn't anticipate during training. I'm also finally done with exams, so I want to take this a step further and try something more challenging. I'm planning to experiment with sim-to-real transfer and see if I can get my robot arm to perform some tasks zero-shot, without any additional real-world training.

You can now book our humanoid cleaners on a recurring subscription, for up to 3 hours per cleaning at tau-robotics.com. Video is first take, uncut, shown at 4× speed. See the reply for detailed footage at 1× speed.

Day 27/365 of robot learning. Today I started implementing PPO from scratch to train a drone to fly towards a target using RL in Isaac Lab. The idea behind PPO is that instead of training on demonstrations like I've been doing with imitation learning, the policy learns through trial and error by interacting with the simulation and maximizing rewards. PPO also clips policy updates to prevent the policy from changing too much at once. The policy observes the drone's position, orientation, and velocities, then outputs continuous control actions for thrust and body torques. I'm planning to experiment more with reward shaping tomorrow, especially adding penalties for unstable flight and exploring curriculum learning inspired by the "Learning to Fly in Seconds" paper. I'll have a video out soon!