Everything we know about Google’s Project Astra so far

Curious about Google's Project Astra? Today we're exploring everything we know about the new multi-modal assistant so far.

Google’s Project Astra on a smartphone

Google’s gone all in on AI assistance. At yesterday’s Google I/O event, the tech giant unveiled Project Astra, its new multi-modal AI assistant. Demis Hassabis, the head of Google’s AI development, hopes it will become a universal assistant, helpful in daily life.

Project Astra aims to make users the focal point of the AI’s efforts. The assistant can see your environment, identify objects, and remember where you left your belongings—a feature that gives it serious clout, in my opinion.

But beyond that, how can Google’s latest AI assistant help us? Here’s everything we know so far:

What is Project Astra?

From the video demo shown at yesterday’s event, Project Astra appears to be an app that sees the world and recognizes objects. In the video, a reviewer asks Astra, “Tell me when you see something that makes sound.” The user scans the room with their smartphone. Astra says, “I see a speaker,” when the camera passes one. It even identifies a specific part of the speaker, the tweeter.

It’s an impressive skill and shows that the bot doesn’t just talk to you but completes tasks for you. Currently, if you want to identify a speaker component, you have to search for an online speaker diagram and hope to find one that looks like yours. From there, you have to read the diagram. It’s a time-consuming hassle compared to working with Astra in realtime.

How does the multi-modal AI operate?

Project Astra is worlds beyond current voice assistants, which respond to vocal requests/commands with just an audio response or by triggering a smart home action. According to Engadget, Astra can process visual information immediately and works from a massive database, possessing a memory, so to speak.

Hassabis says that speeding up the system has been paramount in developing Astra. An AI assistant isn’t much help if it lags. So, the team sped up the module and improved the infrastructure.

What can the AI assistant do?

Identifying the parts of a speaker is a cool trick. But a truly useful AI assistant should go beyond that. For now, even Google isn’t sure how the assistant will work or how it’ll fit into your life. For the moment, the DeepMind team is still working on how to combine the different multimodal models together in the best way.

However, according to The Verge, travel planning is one potential use Google is exploring now. A new tool uses Gemini, Google’s large language model, to create an itinerary for your vacation that you can edit and modify.

Hassabis is also enthusiastic about using AI agents in smart glasses and smartphones. So, perhaps your next pair of Google smart glasses will remind you where you left your keys before you set out for the day.

What are the reviewers’ thoughts on Project Astra?

Reviewers from Engadget believe Project Astra needs some time. During a demo, the reviewers asked the AI assistant to tell a story about objects on a table. While the story was creative and fun, they found it to be a feature that would appeal more to children than something that could make people’s lives easier.

Astra could identify one reviewer’s clothing and remember where he had left his sunglasses. The objects weren’t part of the demo, so it’s the technology is advanced, the potential to save time and stress.

However, there’s much work to be done. The reviewers point out that Astra is cloud-based and can’t run on-device. Also, its ‘memories’ of objects only last for one session. If it’s to work in my house, it’ll need to recall item placement for at least a week.

What does the future hold for Project Astra?

For now, Google has no plans to launch Project Astra. It’s still in the early stages of development. I mentioned earlier that Hassabis likes integrating Astra features into glasses and mobile devices, so maybe we’ll experience it there. But for now, the project is for in-house learning only.

If anything, Project Astra shows how far AI development has come in just a short time. A year ago, we were experimenting with large language model bots’ use in work and daily life. Now, an AI assistant can remember where we left objects in a room. That’s a big jump.

Summary

Overall, Project Astra features will take time to develop fully. However, the project will help the DeepMind team learn more about AI and its potential uses. That way, it can steer development in more targeted ways and better answer the question of how AI can best improve people’s lives.