Designer, Google DeepMind
Joined April 2009
- Tweets295
- Following116
- Followers12.7K
- Likes311
There has been a lot of work on light modeling the bokeh over the years, but I think that one reason people may not like it as much now is that there’s so much tech trying to make the edges “correct” that it loses some of the artistic feel.
Portrait mode photos looked better on iPhone 7 Plus through iPhone X than on the latest iPhones
I remember Phil Schiller telling me that apple really modeled the bokeh after real attributes of real camera lenses. The natural fall off from the eyes and why the ears and the rest of the image is so blurry correctly mimics a really wide aperture lens, like F1.2 or 1.4
It’s really portrait modes on Android phones (especially Pixel) that really screwed everything up. They trained everyone to believe that full segmentation between the subject and background equals a better portrait mode. I get it. Most people do not understand the difference between Gaussian blur and proper depth of field / bokeh from an aperture blade
Johnnie Manzari retweeted
Wonderful recollection by @johnnie of the lessons we learned developing the original Portrait Mode.
Computational photography has come a long way since then. Technologies evolve. Techniques change. But the principles that guide good design remain.
To design for technology that constantly changes, we must design for what doesn’t: how we see.
Portrait mode launched 10 years ago today.
I think it resonated with people because it didn’t feel digital. It felt analog. And this was likely because the inspiration behind its technique went all the way back to oil painting. Sometimes pure math isn’t enough; you need to combine it with insights from the liberal arts.
If you recall Portrait mode shipped a month after the iPhone did. This was because this insight about oil painting took a while to figure out.
Initially, Portrait mode was seen as a giant math problem. You’d do disparity calculations on the two sensors and output a depth map. The problem was this approach feels and looks digital; it feels fake. In areas without texture, it’s impossible to do this calculation. In areas with ultra fine detail, a lower resolution depth map leads to terrible transitions and a full resolution depth map takes forever to compute. One solution would be to reduce the amount of bokeh so the issues are not as obvious. But then you lose the whole point of the feature, which was to make it look like you had a large, wide-open lens.
The solution to this conundrum—the tradeoff between artifacts and the amount of blur— was quite elegant, but it took a while to stumble across. There’s a saying in painting that if you get the eyes right, nothing else matters. Inspired by this principle from oil painting, the idea was to make sure we never got the eyes wrong. We brought in a machine learning block to find the eyes, and draw a mask layer that protected them, then paired that with the disparity depth map. Counterintuitively, the solution was not to decrease the amount of blur, it was to increase it. The additional blur that we could add once the eyes were protected allowed us to hide the artifacts that were create by bad depth calculation. We let the ears go soft, let the nose go a bit soft. This is that wide open look that people loved, even if people didn’t realize that the ears going soft was why.
I was thinking about this recently with regard to AI. We’ll be able to do more amazing things… we just need to study the masters who came before us.
Portrait mode launched 10 years ago today.
I think it resonated with people because it didn’t feel digital. It felt analog. And this was likely because the inspiration behind its technique went all the way back to oil painting. Sometimes pure math isn’t enough; you need to combine it with insights from the liberal arts.
If you recall Portrait mode shipped a month after the iPhone did. This was because this insight about oil painting took a while to figure out.
Initially, Portrait mode was seen as a giant math problem. You’d do disparity calculations on the two sensors and output a depth map. The problem was this approach feels and looks digital; it feels fake. In areas without texture, it’s impossible to do this calculation. In areas with ultra fine detail, a lower resolution depth map leads to terrible transitions and a full resolution depth map takes forever to compute. One solution would be to reduce the amount of bokeh so the issues are not as obvious. But then you lose the whole point of the feature, which was to make it look like you had a large, wide-open lens.
The solution to this conundrum—the tradeoff between artifacts and the amount of blur— was quite elegant, but it took a while to stumble across. There’s a saying in painting that if you get the eyes right, nothing else matters. Inspired by this principle from oil painting, the idea was to make sure we never got the eyes wrong. We brought in a machine learning block to find the eyes, and draw a mask layer that protected them, then paired that with the disparity depth map. Counterintuitively, the solution was not to decrease the amount of blur, it was to increase it. The additional blur that we could add once the eyes were protected allowed us to hide the artifacts that were create by bad depth calculation. We let the ears go soft, let the nose go a bit soft. This is that wide open look that people loved, even if people didn’t realize that the ears going soft was why.
I was thinking about this recently with regard to AI. We’ll be able to do more amazing things… we just need to study the masters who came before us.
The second I saw that transition I thought “here’s the next ‘change the whole industry’ design from Chan”.
I’ve gotten this question enough times that I figured I’d just post it: The iPhone Duo cameras are not as good as your 17 Pro (or 18 Pro). So please do not preorder it then complain to me that you aren’t happy with the low light or sharpness. And yes, Apple could just make an absolutely massive camera bump on the back to resolve this problem, but that was seen as the wrong tradeoff.
This is another example of something that I thought would take years to solve and yet here we are.
Johnnie Manzari retweeted
Asked Gemini 4 Argon for an interactive museum of Apple computers.
...and didn't provide any images or data.
It scraped archival Mac photos, wrote a tool to clean the images, pulled 50 years of specs, and built a 3D Cover Flow with UI that morphs to match the decade you’re in.
Today we are rolling out Gemini 4 Argon globally in the Gemini API, to Pro + Ultra in Gemini App, and Ultra subs in Antigravity!
Argon is a huge step forward for Gemini, and a labor of love. This model has been battle tested by over 100,000 people at Google over the last month, with so much new innovation, ambition, and hard work baked into this release. The team has been heads down over the last 6 months working to make this possible across the stack (research, infra, safety, eng, etc).
We rolled the model out to cyber defenders as part of our fairwind program and early access customers over the last 2 weeks, it has been amazing to see and talk to folks who are now using this model daily. Seeing the models writing capability, raw knowledge and persistence for coding, paired with new features like 1M token output when the model needs to work more has gotten me even more excited about our path ahead.
This is just the start of the Gemini 4 era. As you saw with 3.5, 3.6, 3.7, and 3.8 Flash, our iteration speed with Gemini is fast : ) Now with Gemini 4 Argon to work from, I am cannot wait to see the progress continue, and most importantly see what this new model enables for the world.
This is such a special moment in the arc of technology, as millions of people everyday tackle problems they could never have thought possible before. Gemini 4 Argon was built to help you tackle the hardest problems. I have personally felt this spark with Argon and hope you do too. Onwards!!
After Steve died there was a lot of debate in Silicon Valley around “Who is the next Steve Jobs?” Who would be not just making innovative products, but serving as the spiritual guide for the tech industry, and an elder statesman on difficult social issues. It’s been 15 years, and the more time passes, the more singular Steve appears to be.
Steve Jobs died on October 5, 2011
Today marks 15 years since he died.
Couple years ago Apple created a book titled “Make Something Wonderful” to celebrate his life, and gave it to Apple employees.
I got the physical and digital versions of the book, would you like to see more photos?
When people think of Steve Jobs they think of Apple
But he’s done hardware, software, and even movies, without him Pixar and the new era of animations wouldn’t have become what it is today.
He released Toy Story movie in 1995.
Steve passed away in his home in Palo Alto, California at the age of 56, because his pancreatic neuroendocrine tumor came back.
Steve refused to go through the chemotherapy, he believed that his tumor was discovered early enough in 2003, that vegan/fruitarian diet, acupuncture, herbal remedies, and juice fasts were enough to get rid of it
Eventually in 2004 he had a surgery to remove it, but believed he didn’t need chemotherapy and the surgery was enough.
He kept saying that he’s cancer free and got a second life when doctors believed he needed chemotherapy or radiation or else his tumor may come back.
RIP Steve Jobs.
The best product people have intense clarity in their point of view. I remember a meeting many years ago where people took Jony Ive through the state of the art foldable display technology for that time. This was back with plastic-feeling screens that had a big crease down the middle. His feedback was simply: “Your idea is a big display. But this isn’t a great display. So you don’t have an idea.” It might have taken about 10 years after that to make a great foldable display— nano texture, updated hinge mechanisms, behind display camera.
This single transition says so much about how Apple approaches a problem. Other companies copied the aspect ratio, more or less, but because they didn’t understand the motivation and reasoning behind it, they couldn’t copy the essence of what makes it so successful. If you have this aspect ratio, and do a lot of work in software, it’s no longer two products glued together, it appears as one seamless, continuous product. This is complicated to do when the inside display is a different display technology, with a different bezel thickness, and a camera in a different location (hence putting it behind the display on the inside). You don’t begin with a product size and shape, you begin with the idea that this is one singular product and make every decision in service of that idea.
This single transition says so much about how Apple approaches a problem. Other companies copied the aspect ratio, more or less, but because they didn’t understand the motivation and reasoning behind it, they couldn’t copy the essence of what makes it so successful. If you have this aspect ratio, and do a lot of work in software, it’s no longer two products glued together, it appears as one seamless, continuous product. This is complicated to do when the inside display is a different display technology, with a different bezel thickness, and a camera in a different location (hence putting it behind the display on the inside). You don’t begin with a product size and shape, you begin with the idea that this is one singular product and make every decision in service of that idea.
At a lot of companies, the internal concepts look good but what ships lacks the same level of craft. This iPhone Duo transition is even better in the shipping form than what it looked like internally when I last saw it. This isn’t luck or something that occurs easily; people are putting in long hours to make this happen.
At a lot of companies, the internal concepts look good but what ships lacks the same level of craft. This iPhone Duo transition is even better in the shipping form than what it looked like internally when I last saw it. This isn’t luck or something that occurs easily; people are putting in long hours to make this happen.
Steve Jobs’ greatest product wasn’t the iPhone or the Mac, it was Apple itself: the machine that makes the machine. When Tim Cook took over, conventional wisdom was that Apple would slide into irrelevance, because Steve’s system required a singular product visionary driving the entire company. I would argue Tim’s greatest product and legacy isn’t the Watch or Services or scaling the iPhone, it was showing how to run a disciplined, top-down company with a team vs an individual. Maybe if his talent in operations was scaling, you could say he not only scaled the products, he showed how to scale the company, too.
Sugar Bowl found my iPhone that I lost in December snowboarding. It was sitting under the Disney lift for 7 months buried snow and sitting in the sun. Not a scratch on it. Works perfectly. Incredible.
User interfaces are about to go through another big transformation, on the scale of command line to GUI, if not bigger. The new tools aren’t about making the old designs more efficiently, they are about making entirely new experiences.
Johnnie Manzari retweeted
One of my first projects as an EPM. The tape length being accurate in each spool 👌
I’ve been using this for UI design, and it’s incredible. I’m old enough to remember when getting a video to loop seamlessly was a whole project unto itself.
I’ve joined the Google DeepMind design team. I’ve spent my professional career working as a toolmaker, and while I’ve been lucky to have contributed to some incredible products in the past, I may well look back on the work here as the most important when it comes to building tools that are helpful, delightful, and genuinely enriching to people’s lives.