VLX — Edge-native multimodal AI for real-time perception, reasoning, and action. GitHub: https://xitter.cf/t.co/ZUYXqCMHAR Discrod: https://xitter.cf/t.co/NJjOYSxO03

Joined October 2024
730ms changes what you can build with a 3B vision model. VLX-Seek 3B now reaches ~730ms end-to-end latency in our current test setup: NVIDIA T5000 / MAXN 1920×1080 image 27-token output But the interesting part isn’t the benchmark number itself. It’s what happens when natural-language visual grounding starts responding at sub-second speed. You can ask: “Find the worker standing next to the machine.” “Locate the damaged package on the conveyor.” “Find the red vehicle near the entrance.” And get the target region back fast enough to plug into a larger workflow. That opens up more practical use cases across: • Robotics • Industrial inspection • Visual agents • Surveillance analysis • Edge vision systems For visual AI, accuracy tells you whether the model can find the target. Latency determines how usable that ability actually feels.
3
369
What if machine vision could find ANYTHING you describe? We put VLX-Seek through 3 quick tests: 🔩 Count the screws 🔍 Separate bent fasteners from straight ones ⚙️ Find the yellow gear No fixed class list. No retraining for every new object. Just tell VLX-Seek what to look for. From object counting to fine-grained visual grounding, all through natural language.
2
1
270
Every orange counts. 🍊 VLX Seek brings fruit detection to the production line, identifying oranges among leaves and overlapping fruit. It’s a foundation for automated counting, flow monitoring, and smarter sorting—turning what cameras see into data teams can use.
1
2
1
3
444
One optimization won’t get a vision model to sub-second latency. The full stack matters. For VLX-Seek 3B, we tested each optimization independently against the same vLLM baseline (~1.54s). Only one optimization was enabled at a time. All other components remained on the baseline vLLM setup. Here's how each performed: Vision Engine (TensorRT): ~1.14s LM W4A4: ~1.28s FlashInfer GDN Prefill: ~1.51s LM Head W4A16 only: ~1.46s Each optimization targets a different bottleneck, but none alone breaks the 1-second barrier. Now enable them all together: ~1.54s → ~730ms. The full stack optimizes every major stage of inference: Vision Processing → Prefill → LM Linear Layers → LM Head No single magic trick. Just removing bottlenecks, stage by stage. Faster visual grounding starts with optimizing the entire pipeline.
4
3
287
What if a sorting line could understand exactly which package to pick? 📦🤖 With VLX-Seek, a robotic arm can locate packages directly from natural-language instructions: “Find the damaged parcel.” “Pick the box with the red label.” “Locate the package on the left conveyor.” VLX-Seek identifies and localizes the target in the scene, then passes the position to the robotic arm for sorting. From visual grounding → robotic picking → automated sorting, one instruction can drive the whole workflow. No fixed class list. No manually defined target categories. #Robotics #ComputerVision #VLM #Automation #VLX
1
3
207
How much does W4A4 actually speed up VLX-Seek 3B? ⚡ We tested multiple LM backbone quantization setups on the same NVIDIA T5000: ⏱️ Inference Latency: - Baseline PyTorch: ~1.73s - LM backbone W4A16: ~1.42s - LM backbone W4A4: ~1.28–1.29s The W4A4 setup covers major LM backbone linear layers, including: • Attention q/k/v/o projections • MLP gate/up/down projections • Linear Attention projections The goal isn't just fewer bits. It's reducing the computational cost of operations repeated throughout inference. 🚀 Combined with vision-side and prefill optimizations, latency drops even further. Quantization is one piece of the speed puzzle — not the whole stack.
2
200
👷 Can AI instantly spot who's wearing a helmet on a busy construction site? With VLX-Seek, just give a simple instruction: "Find all workers wearing a helmet" ⚡️ Seconds-level response 🎯 Accurate visual grounding 🏗️ Built for real-world construction site monitoring No fixed categories. Just natural language → precise detection. Smarter visual monitoring starts with VLX-Seek.
1
126
Can a vision model keep up with a fast-moving production line?⚡️ VLX-Seek can detect and locate bottled water in real time, even in dense industrial scenes. Just describe what you’re looking for, and it finds the right bottles fast and accurately — built for practical visual inspection on the edge.
1
115
Where does the latency of a vision model actually come from? Not just the language model. For VLX-Seek 3B, we moved the main and auxiliary vision towers to a TensorRT-based vision engine. Same test setup: NVIDIA T5000 / MAXN 1920×1080 input 27-token output Latency dropped from: ~1.73s → ~1.14s That’s already a major reduction before combining the rest of the optimization stack. A visual grounding model has to process the image before it can reason about what and where to find. So if the vision pipeline is slow, optimizing the LM alone won’t get you to low latency. For VLX-Seek, faster grounding starts with making the model see faster first.
2
1
4
197
730ms. On a 1080p image. ⚡ VLX-Seek 3B just got 2.36× faster. Tested on an NVIDIA T5000 with a 1920×1080 image and 27-token output: • PyTorch baseline → ~1.73s • vLLM → ~1.54s • Fully optimized → ~730ms That’s roughly 58% lower end-to-end latency. What changed? • TensorRT vision engine • FlashInfer GDN prefill • W4A4 quantization for LM linear layers • W4A16 LM head optimization The result: VLX-Seek 3B now reaches sub-second visual grounding on 1080p input. That matters when the model isn’t just answering a benchmark — it’s sitting inside a robot, an inspection pipeline, a visual agent, or an edge vision system waiting to find the next target. From 1.73s to 730ms. And it isn’t the end...
2
1
3
141
Try it out on om-agent.com
🤖 Made with AI
1
66
On a fast-moving production line, every second counts. ⚡ Built for edge devices, VLX-Seek combines language-guided localization, counting, and fine-grained identification—reducing reliance on cloud transfers and unstable networks. From a simple instruction to locating the targets that matter—in seconds.👇
2
138
Thanks to @ThePirateFace for adding VLX-Seek to their model collection! 🏴‍☠️ VLX-Seek brings fine-grained visual perception to embodied AI—locating targets described in natural language, distinguishing specific instances, and recognizing when a target is absent.
2
6
362
🚗 Cars, buses, animals and more — what can VLX-Seek find? We put together a collection of real-world object detection demos to show VLX-Seek in action. From everyday objects to complex scenes, just describe what you're looking for and let the model locate it in the frame. Less time defining categories. More freedom to explore. 👀
2
98
Can AI automatically inspect a production SOP — and catch a missed step? 👀 We tested VLX-VR on a full baking workflow — no timestamps, no step labels. It can: • Track each step in order • Detect skipped steps • Spot visible mistakes It moves from process-level understanding to fine-grained detail judgment, as it can even: • Read ruler measurements • Tell which oven rack is being used • Output the inspection as structured JSON From baking training to production-line SOP inspection, VLX-VR goes beyond what happened. It checks whether the process was actually followed correctly.
2
109
A crowded herd. One prompt. Results in seconds. ⚡ “Find all the cattle🐄 in the field.” VLX-Seek quickly locates the targets with precise visual grounding — even in dense, real-world scenes. Fast response. Accurate localization. Natural-language input. Built for visual tasks where every second counts.
2
183
On-device multimodal AI is moving beyond “smaller models.” The real shift is happening at the capability level: 🔍 Find the right object 🎥 Understand a live visual stream 🧠 Reason over what is happening ⚡ Respond where the data is generated Across the industry, more multimodal intelligence is moving closer to the device. That’s also the direction we’re building toward with VLX. From perception, to reasoning, to action, the edge is becoming much more capable.
1
106
🚜Track the yellow forklift. Follow every move. With VLX-Seek, simply identify the yellow forklift and track its location as it moves through the warehouse. No predefined classes. Just natural language and precise visual grounding. From detection to tracking, every move stays in sight. 👀
1
126