Google has expanded AI Mode to include multimodal capabilities, now available to more Labs users in the U.S. Following positive feedback from Google One AI Premium subscribers, who appreciated the design and response time, AI Mode is being utilized for complex queries, such as product comparisons and trip planning.
The new multimodal feature allows users to snap a photo or upload an image, enabling them to ask questions about the visual content. This integration combines visual search capabilities from Lens with a customized version of Gemini, enhancing the understanding of images by analyzing the relationships between objects, their materials, colors, and arrangements.
AI Mode utilizes a query fan-out technique to issue multiple queries about the entire image and its components, resulting in nuanced and contextually relevant responses. For instance, it can identify books on a shelf, provide recommendations, and facilitate follow-up questions for more refined searches. Google continues to test and improve AI Mode, encouraging user feedback through the Labs program.
