Google‘s AI-fueled Ask Photos feature, previewed at Google I/O 2024, has begun initial testing, as first shared by 9to5 Google. Ask Photos employs Google’s Gemini AI to search a user’s photo library using natural language without first organizing and tagging the image. Google hasn’t shared any release date, but it looks like a select group of users are being asked to try it out and offer feedback before a broader rollout.
Ask Photos essentially expands Gemini’s ability to parse visual information from a user’s photos. The AI can then comb through images, presumably those in an album or on a device that the AI has permission to access and search through. It’s not just a keyword hunt either, as the AI can also answer questions posed by the user. For instance, at the initial demonstration at Google I/O, CEO Sundar Pichai showed how he could use it to ask his phone for his license plate number, with the answer provided thanks to a photo of the license plate in his album.
The version highlighted by 9to5 Google was discovered by a user on his Pixel 8 Pro. Unlike the official demo, where Ask Photos had its own tab, the feature now seems to be available on the Search tab via an “Ask” button next to the search bar. Tapping the button brings up Ask Photos as a new interface, including sample prompts and space to write your own response to the question, “What would you like to see?” Some aspects of the feature are still undergoing testing, though, as the “Photos of me over time” sample prompt brought back a technical error response and a suggestion to “Use classic search.” Regardless, the feature apparently disappeared shortly after the user first saw it.
Google Ask Photos reading license plate numbers out of the photo library#GoogleIO pic.twitter.com/ElSEQHrvVwMay 14, 2024
Visual Search
Employing Gemini to look through photos obviously benefits users, especially those who have a lot of poorly organized images. Google Photos has some ability to find photos of specific events and people, but applying Gemini’s AI models would vastly improve the platform’s abilities. The AI could far better discern what photos are in the same place, identify people across different time periods, and otherwise organize images based on detailed criteria.
Ask Photos is ultimately just another facet of Google’s work to embed Gemini into every product and service it offers. Of course, the feature’s apparent limitations and technical issues at the moment highlight the complexity of doing so. But, when combined with related rollouts like making Gemini available from the lock screens of Android phones, the bigger picture becomes clear, even without asking Gemini for help.