Google's Project Astra Is an AI Assistant That Can Respond to What It Sees


At I/O 2024, Google made lots of exciting AI announcements—but one that has everyone talking is Project Astra. Essentially, Project Astra is what Google is calling an "advanced seeing and talking responsive agent." This means that a future Google AI will be able to get context from what's around you and you can ask a question and get a response in real time. It's almost like an amped-up version of Google Lens.

Project Astra is being developed by Google's DeepMind team, which is on a mission to build AI that can responsibly benefit humanity; this project is just one of the ways it's doing so. Google says that Project Astra is built upon its Gemini 1.5 Pro, which has gained improvements in areas such as translation, coding, reasoning and more. As part of this project, Google says they've developed prototype AI agents that can process information even faster by continuously encoding video frames and combining video and speech input into a timeline of events. The company is also using their speech models to enhance how its AI agents sound, for a wider range of intonations.

Google released a two-part demo video to show off how Project Astra works. The first half of the video shows Project Astra running on a Google Pixel phone; the latter half shows the new AI running on a prototype glasses device.

Lifehacker Logo
You May Also Like

In the demo video, we can see the user using their Pixel phone with a camera viewfinder open and moving their device around the room while asking the next-generation Gemini AI assistant, "Tell me when you see something that makes sound" and the AI responding by pointing out the speaker on the desk. Other examples in the video include asking what a part of the code on a computer screen does, what city neighborhood they're currently in and coming up with a band name for a dog and its toy tiger.

What do you think so far?

While it will be a long time before we see this next-generation AI from Project Astra coming to our daily lives, it's still quite cool to see what the future holds.

Recommended Articles

About Us

We are a dynamic information platform dedicated to delivering timely, relevant, and reliable content across a broad spectrum of topics. From the latest in technology and business to lifestyle, health, and global affairs, we aim to keep our readers informed and inspired.

Our editorial team is committed to maintaining high standards of accuracy and clarity, ensuring that every article provides value and context in an ever-changing world. We believe in the importance of accessible knowledge and strive to make complex topics understandable for everyone.

Whether you're here to stay updated on current events, explore thought-provoking features, or simply learn something new, our goal is to offer a trustworthy source of information that meets the needs of a diverse and curious audience.