Multimodal Generalist, DS @HiLabs_inc | GSoC ’26 @GNUOctave | @iitroorkee alum | Explaining SOTA DL & systems | Constantly building, lifting, running.
Bengaluru, India
Joined June 2023
- Tweets2.6K
- Following1.3K
- Followers1.2K
- Likes10.2K
would be grinding ML System design today, from 4pm to 6am, winter arc going good, would post the learnings post that
2
9
354
consolidated my findings in this blog was heck of a learning and experimentation, loved it @WeAreSwiggy , @letsblinkit , @instamart_it , @ZeptoNow 😛😛 would love to improve your systems as a DS / AI engineer if yall give me a chance, can DM you my resume
for @letsblinkit it plateaus cause I suspect the scroll ends after a short length. Else in my experiments as well the memory usage by the app was climbing, both @zomato as well as @letsblinkit. For me the spike was much less as compared to what you report but I think it also depends on the iphone one is using as well as the battery percentage (during the time of experiment my battery percentage was around 43%, I gave sufficient time to cool down in both runs) and was my ios version is 26.6.1
4
14
457
but apparently it doesn’t work with other regional languages of india. It says AI search handles it handles languages with no transliteration step needed but, it isn’t the case on my iphone.
from what i can think off is that, a short product word in native script seems to go through the phone’s translator before it even reaches the server. And for kannada and telugu, Apple simply has no translator.
Mp what you are doing is having an LLM and making it generate keywords against which exact keywords searches + vector searches are being done, and since it works on meaning rather than exact words it handles Hindi / Kannada or Hinghlish sentences easily.
Nice work by @WeAreSwiggy but can you look into this issue as well? Cause technically it should be working
We upgraded #Swiggy Instamart with LLM-Powered Search! It understands intent, builds smart shelves, & reads Hindi, Hinglish, Kannada, & Telugu.
Read how we built it on Swiggy Bytes: bit.ly/4zwtY3d
#WeAreSwiggy #SwiggyBytes
3
190
for @letsblinkit it plateaus cause I suspect the scroll ends after a short length. Else in my experiments as well the memory usage by the app was climbing, both @zomato as well as @letsblinkit. For me the spike was much less as compared to what you report but I think it also depends on the iphone one is using as well as the battery percentage (during the time of experiment my battery percentage was around 43%, I gave sufficient time to cool down in both runs) and was my ios version is 26.6.1
does anyone at @zomato actually use their own app
it's been cooking my iphone for 6+ months. every update i thought now they'll sort it out but they never did. so i got fed up and profiled it myself, hooked it up to xcode instruments. in about 2 mins of scrolling, the app goes from 220 mb to almost 900 mb of memory. ios's own heat meter hits "serious" (the level where it can start slowing your phone down) and it crashed 4 times while I was profiling it
blinkit on the other hand sits flat at 300 mb. maybe you guys should talk to each other
so there are only 2 reasons this has survived 6 months of updates:
> you don't see it in your telemetry (you track every scroll i make, so no)
> nobody cared enough to look into it
@deepigoyal which one is it
well from what I think, in order to train any kind of recommendation engine its quite fundamental to store the clicks, scrolls just like we do in time series based sequences, and this behaviour is understood by Neural Networks and hence we can in runtime optimise the suggestions being shown to someone / improve the recommendstions altogether, but all these has to be stored somewhere and I think this is some kind of eviction issue which can be solved easily, its not that big of a problem for blinkit as its initial scroll page is small whereas in zomato it’s very lengthy and hence the explosion
1
34
for @letsblinkit it plateaus cause I suspect the scroll ends after a short length. Else in my experiments as well the memory usage by the app was climbing, both @zomato as well as @letsblinkit. For me the spike was much less as compared to what you report but I think it also depends on the iphone one is using as well as the battery percentage (during the time of experiment my battery percentage was around 43%, I gave sufficient time to cool down in both runs) and was my ios version is 26.6.1
11
I used one iPhone 16 (iOS 26.6.1) connected to my Mac, with the normal App Store versions of each app and one account. Nothing inside the apps was modified; they were used the way a customer uses them. An open-source helper, XCode, WebDriverAgent, let the Mac tap, type and swipe on the phone, and another, pymobiledevice3, read each app’s memory, CPU and frame rate
For search, I typed 25 everyday grocery searches into each app’s own search box, plus three other ways of writing each one (a typo, Hindi script, Hinglish or a pack size), and saved the results screen: 375 searches across Blinkit, Zepto and Instamart. For memory, I opened each app fresh and scrolled its home feed for five minutes while recording memory and smoothness, letting the phone cool between apps
I was playing around the @letsblinkit app’s search and recommendation engine and compared it with @instamart_it , @ZeptoNow and here are my findings. First the negatives and the the positives, I would also be suggesting fixes in the end :)
1
4
288
I was playing around the @letsblinkit app’s search and recommendation engine and compared it with @instamart_it , @ZeptoNow and here are my findings. First the negatives and the the positives, I would also be suggesting fixes in the end :)
2. Mapping of “devanagari” words to hinglish (like दूध out as “doodh” and then hopes its Hinglish word list knows it, but zepto directly converts it to “milk”, this way Zepto looses less items)
I used each app’s own search box, typing the way a person does, from one fixed address. The idea is simple: “doodh”
should find the same milk as “milk”. If a rewritten search loses the product, something in the search pipeline broke, and the kind of rewrite tells us which part
2
60
1. Pack sizes get lost, Ask Blinkit for “rice 1kg” and its banner says it searched for “rice”; a 5 kg bag comes first, in all four ways I phrased it ( same issue is with instamart, but zepto gets this right every single time)
2
110
Some of the nice takeaways from today's sprint:
> Think of order in which things are sent to the agent? to maximise prompt caching? (as there are tool definitions, tool results, history, context injection, system prompts etc)
> Agent degradation is a gradient not a cliff
> Can have a day-dreaming type scheduled job/mannual job which can run at fixed intervals and fix the typed stores from the logged transcripts as @AnthropicAI suggested
> Keep a check on the number of tools an agent has access to as post 30, most tool definitions might overlap and at around 100-200 its more of a bloat, so dynamically allow the agent to explore them using retrieval
> explore offloading, have the summarisation, typed things offloaded to in memory if needed and keep a pointer to that location where agent might access it as suggested in @AnthropicAI 's blog
> in multi-agent architecture it's not advisable to allow all the sub-agents to write in the same file as they might overwrite each other
> multi-agent architecture solve context bloating efficiently but then comes the coordination problem as each comes with there independent context
> a day dreaming job can prune, merge typed stores even after the CRUD based typed stores update this way we keep poisoning away
> garbage-in = garbage-out, so also focus on the least looked upon part in RAG, query re-writing, then query expansion, sub-queries generation for complex ones and then retrieval
> chunking strategies we can have @typesafeai JEV based classification (just like we do in agentic chunking) to handle different things differently, as for example it's not logical to split HTML, Code the same way to do for docs
> late chunking, what it is and what it isn't is imp to understand
Gonna read these blogs around agentic memory and context management, evaluation of retrieval systems and memory layers, will post an update by night UwU:
1. Anthropic's guide on agent memory anthropic.com/research/compo…
2. Langchain docs on memory: docs.langchain.com/oss/pytho…
3. Late chunking from @JinaAI_ and @weaviate_io jina.ai/news/late-chunking-i…, jina.ai/news/what-late-chunk… and arxiv.org/pdf/2409.04701, weaviate.io/blog/late-chunki…
4. See how needle in the hay datasets are structured for agentic retrieval benchmarking
Please find the article here: teapaw.org/beingamanFF/status/212… UwU
Gonna read these blogs around agentic memory and context management, evaluation of retrieval systems and memory layers, will post an update by night UwU:
1. Anthropic's guide on agent memory anthropic.com/research/compo…
2. Langchain docs on memory: docs.langchain.com/oss/pytho…
3. Late chunking from @JinaAI_ and @weaviate_io jina.ai/news/late-chunking-i…, jina.ai/news/what-late-chunk… and arxiv.org/pdf/2409.04701, weaviate.io/blog/late-chunki…
4. See how needle in the hay datasets are structured for agentic retrieval benchmarking
12
657
Gonna read these blogs around agentic memory and context management, evaluation of retrieval systems and memory layers, will post an update by night UwU:
1. Anthropic's guide on agent memory anthropic.com/research/compo…
2. Langchain docs on memory: docs.langchain.com/oss/pytho…
3. Late chunking from @JinaAI_ and @weaviate_io jina.ai/news/late-chunking-i…, jina.ai/news/what-late-chunk… and arxiv.org/pdf/2409.04701, weaviate.io/blog/late-chunki…
4. See how needle in the hay datasets are structured for agentic retrieval benchmarking