AI Engineer | IITian
multiverse
Joined June 2020
- Tweets233
- Following81
- Followers78
- Likes82.3K
Pinned Tweet
My phone gallery is almost 60% screenshots, memes and random forwards from whatsapp. So I built an app that scans your photos and finds them for you (to delete)!
Introducing Photokeeper: extremely lightweight on device photo organizer tool. Fully open source. Runs on anything.
<10 ms per photo. 2.6 MB model. Runs fully offline, nothing leaves your device.
The Exponential Time Hypothesis states that there exists no algorithm that can solve the 3SAT problem in less than exponential time. This is one of the fundamental assumptions computing is built upon. This is because 3SAT is a NP complete problem, which means it can be solved in polynomial time by a non deterministic Turing machine (NTM). But since NTM is a mathematical abstraction, so it is accepted that these problems are practically unsolvable in polynomial time by a computer, which are deterministic Turing machines. This hypothesis is considered true and a fundamental basis of computing research. For example, many security algorithms depend on the "hardness" of these problems.
If existence of the algorithm is proved, it is only a matter of time before such an algorithm is discovered. This would mean a large number of highly researched and unsolved problems will now have a efficient implementable solution and will be a world changing breakthrough.
How does Photokeeper achieve near instant image classification even on edge devices?
The trick is we are not using one single large image model, instead we are using a 5-step pipeline with each step scoring a single aspect of the image
A thread 🧵
My phone gallery is almost 60% screenshots, memes and random forwards from whatsapp. So I built an app that scans your photos and finds them for you (to delete)!
Introducing Photokeeper: extremely lightweight on device photo organizer tool. Fully open source. Runs on anything.
<10 ms per photo. 2.6 MB model. Runs fully offline, nothing leaves your device.
Its uses a weighted vote to combine each of these pipeline step outputs. Each signal is turned into a keep-likeness score between 0 and 1. The scores are multiplied by their weights (which sum to 1) and added up. If the total is above the threshhold the photo is kept, otherwise it's marked for deletion.
This is how the pipelines run, but there are some nontrivial optimizations possible on this pipeline that would make the processing extremely fast and memory efficient for actual edge devices. I will post about those optimizations in the future threads. In the meantime, take a look at readme for more details: github.com/aritroCoder/photo…
When compared on a 1000 image dataset:
• clip-ViT-B-32: 91.6% accuracy, 51 ms/img, 1.9 GB RAM, ~600 MB model
• PhotoKeeper: 90.4% accuracy, 9.5 ms/img, 171 MB RAM, 2.6 MB model
Photokeeper is Recall tuned, so we tradeoff accuracy for higher recall. Because you review the list before deleting, and missing junk is worse than over-flagging.
Under the hood it's a C++ engine with a stripped-down static OpenCV build, and the UI is Flutter, so one codebase covers 5 platforms. Engine repo for the curious: github.com/aritroCoder/photo…
Same repo contains both the C++ and python version of the engine.
