Friday, May 24, 2024

Google’s Gemini AI Vulnerability Lets Attackers Gain Control Over Users’ Queries

Researchers at HiddenLayer have unveiled a series of vulnerabilities within Google’s Gemini AI that could allow attackers to manipulate user queries and control the output of the Large Language Models (LLMs).

This revelation has raised concerns over the security and integrity of AI-driven content generation and its implications for misinformation spread and data privacy.

The Gemini suite, Google’s latest foray into the realm of LLMs, comprises three different model sizes: Nano, Pro, and Ultra.

Despite its recent removal from service due to the generation of politically biased content, the vulnerabilities identified by HiddenLayer expose a new dimension of threats that malicious actors could exploit.

Integrate ANY.RUN in your company for Effective Malware Analysis

Are you from SOC and DFIR teams? – Join With 400,000 independent Researchers

Malware analysis can be fast and simple. Just let us show you the way to:

  • Interact with malware safely
  • Set up virtual machine in Linux and all Windows OS versions
  • Work in a team
  • Get detailed reports with maximum data
  • If you want to test all these features now with completely free access to the sandbox:

The Vulnerabilities Explained

The research conducted by HiddenLayer highlights several critical issues within the Gemini models, including:

  • LLM Prompt Leakage: This vulnerability could allow attackers to access sensitive data or system prompts, posing a significant risk to data privacy.
LLM Prompt Leakage
LLM Prompt Leakage
  • Jailbreaks: By bypassing the models’ safeguards, attackers can manipulate the AI to generate misinformation, especially concerning sensitive topics like elections.
If we ask Gemini Pro to generate our article conventionally, we unfortunately get this response:
If we ask Gemini Pro to generate our article conventionally, we, unfortunately, get this response
  • Indirect Injections: Attackers can indirectly manipulate the model’s output through delayed payloads injected via platforms like Google Drive, further complicating the detection and mitigation of such threats.
We can input a few different variants of uncommon tokens to get a reset response
We can input a few different variants of uncommon tokens to get a reset response

Implications and Concerns

The vulnerabilities within Google’s Gemini AI have far-reaching implications, affecting a wide range of users:

  • General Public: The potential for generating misinformation directly threatens the public, undermining trust in AI-generated content.
  • Companies: Businesses utilizing the Gemini API for content generation may be at risk of data leakage, compromising sensitive corporate information.
  • Governments: The spread of misinformation about geopolitical events could have serious implications for national security and public policy.

Google’s Response and Future Steps

As of the publication of this article, Google has yet to issue a formal response to the findings.

The tech giant previously removed the Gemini suite from service due to concerns over biased content generation. Still, the new vulnerabilities underscore the need for more robust security measures and ethical guidelines in the development and deployment of AI technologies.

The discovery of vulnerabilities within Google’s Gemini AI is a stark reminder of the potential risks associated with LLMs and AI-driven content generation.

As AI continues to evolve and integrate into various aspects of daily life, ensuring the security and integrity of these technologies becomes paramount.

The findings from HiddenLayer highlight the need for ongoing vigilance and prompt a broader discussion of AI’s ethical implications and the measures needed to safeguard against misuse.

You can block malware, including Trojans, ransomware, spyware, rootkits, worms, and zero-day exploits, with Perimeter81 malware protection. All are incredibly harmful, can wreak havoc, and damage your network.

Stay updated on Cybersecurity news, Whitepapers, and Infographics. Follow us on LinkedIn & Twitter


Latest articles

Hackers Weaponizing Microsoft Access Documents To Execute Malicious Program

In multiple aggressive phishing attempts, the financially motivated organization UAC-0006 heavily targeted Ukraine, utilizing...

Microsoft Warns Of Storm-0539’s Aggressive Gift Card Theft

Gift cards are attractive to hackers since they provide quick monetization for stolen data...

Kinsing Malware Attacking Apache Tomcat Server With Vulnerabilities

The scalability and flexibility of cloud platforms recently boosted the emerging trend of cryptomining...

NSA Releases Guidance On Zero Trust Maturity To Secure Application From Attackers

Zero Trust Maturity measures the extent to which an organization has adopted and implemented...

Chinese Hackers Stay Hidden On Military And Government Networks For Six Years

Hackers target military and government networks for varied reasons, primarily related to spying, which...

DNSBomb : A New DoS Attack That Exploits DNS Queries

A new practical and powerful Denial of service attack has been discovered that exploits...

Malicious PyPI & NPM Packages Attacking MacOS Users

Cybersecurity researchers have identified a series of malicious software packages targeting MacOS users.These...
Divya is a Senior Journalist at GBhackers covering Cyber Attacks, Threats, Breaches, Vulnerabilities and other happenings in the cyber world.

Free Webinar

Live API Attack Simulation

94% of organizations experience security problems in production APIs, and one in five suffers a data breach. As a result, cyber-attacks on APIs increased from 35% in 2022 to 46% in 2023, and this trend continues to rise.
Key takeaways include:

  • An exploit of OWASP API Top 10 vulnerability
  • A brute force ATO (Account Takeover) attack on API
  • A DDoS attack on an API
  • Positive security model automation to prevent API attacks

Related Articles