Beware of promptware: How researchers broke into Google Home via Gemini

Maria Diaz/ZDNET

ZDNET's key takeaways

• Researchers demonstrated a way to hack Google Home devices via Gemini.
• Google put additional safeguards in place for Gemini in response.
• Keeping your devices up-to-date on security patches is the best protection.

The idea that artificial intelligence (AI) could be used to maliciously control your home and life is one of the main reasons why many are reluctant to adopt the new technology -- it's downright scary. Almost as scary as having your smart devices hacked. What if I told you some researchers just accomplished that?

Also: Why AI-powered security tools are your secret weapon against tomorrow's attacks

Cybersecurity researchers from multiple institutions demonstrated a major vulnerability in Google's popular AI model, Gemini. They launched a controlled, indirect prompt injection attack -- aka promptware -- to trick Gemini into controlling smart home devices, like turning on a boiler and opening shutters. This is a demonstration of an AI system causing real-world, physical actions through a digital hijack.

How the attack worked

A group of researchers from Tel Aviv University, Technion, and SafeBreach created a project called "Invitation is all you need." They embedded malicious instructions into Google Calendar invites, and when users asked Gemini to "summarize my calendar," the AI assistant triggered pre-programmed actions, including controlling smart home devices without the users' asking. 

The project is named as a play on words from the famous AI paper, "Attention is all you need," and triggered actions like opening smart shutters, turning on a boiler, sending spam and offensive messages, leaking emails, starting Zoom calls, and downloading files.

These pre-programmed actions were embedded using the indirect prompt injection technique. This is when malicious instructions are hidden within a seemingly innocent prompt or object, in this case, the Google Calendar invites.

How this affects you 

It's worth noting that, even if the impact was real, this was done as a controlled experiment to demonstrate a vulnerability in Gemini; it was not an actual live hack. It's a way to demonstrate to Google that this could happen if bad actors decided to launch such an attack. 

Also: 8 smart home gadgets that instantly upgraded my house (and why they work)

In response, Google updated its defenses and implemented stronger safeguards for Gemini. These include filtering outputs, requiring explicit user confirmation for sensitive actions, and AI-driven detection of suspect prompts. The latter is potentially problematic since AI is vastly imperfect, but there are things you can do to further protect your devices from cyberattacks.

What you can do to protect your devices

While this attack was launched with Gemini and Google Home, the following recommendations are good ways to protect yourself and your devices from bad actors.

• Limit your permissions within your smart home application. Don't give Gemini, Siri, or other smart home assistants control of sensitive devices unless you need to. For example, I let Alexa access my cameras but don't let the voice assistant control my smart locks.
• Be mindful of the services that you connect with Gemini and other voice assistants. The more devices and apps you connect to your AI assistant (like Gmail, your calendar, etc), the more potential entry points would-be attackers have. 
• Watch for unexpected behavior from your devices and AI assistants and, if something seems off, revoke permissions and report it.

Also: Best antivirus software: My favorites, ranked, for personal device security

As a rule of thumb, you should always keep your devices and apps up-to-date with the latest firmware updates. This ensures that you get the latest security patches to ward off attacks.

Want more stories about AI? Sign up for Innovation, our weekly newsletter.