Threat Actors Exploit AI and LLM Tools for Offensive Cyber Operations

A recent report from the S2W Threat Intelligence Center, TALON, sheds light on the escalating misuse of generative AI and large language models (LLMs) by threat actors on the dark web for malicious cyber operations.

As LLMs like ChatGPT, Claude, and DeepSeek grow in capability, they are increasingly weaponized as offensive tools for exploit generation, malware creation, and bypassing security mechanisms.

Dark web forums such as Cracked and BreachForums are rife with discussions on fine-tuning models, prompt engineering, and leveraging open-source research for nefarious purposes.

This trend underscores a critical shift in the cyber threat landscape, where AI is no longer just a defensive asset but a potent attack vector.

Growing Abuse of Generative AI

Delving into specific incidents, the report highlights a case from January 24, 2025, where a user named “KuroCracks” on the Cracked forum shared an open-source scanner for CVE-2024-10914, a remote code execution vulnerability ideal for botnet creation.

The tool, reportedly developed with AI assistance, included Masscan-based automation code, demonstrating how LLMs, when manipulated to evade safety protocols, can generate exploit-like responses.

Techniques to bypass these guardrails are openly circulated on the dark web, amplifying the risk. Furthermore, threat actors are not limiting their focus to using AI as a tool; they are targeting LLM infrastructure itself.

In February 2025, a user “MTU1500Tunnel” on BreachForums advertised an exploit for the Gemini API, claiming it could manipulate account balances and jailbreak restrictions.

Additionally, tools like WormGPT, an LLM fine-tuned for malicious intent, are promoted as “no-restrictions” alternatives, revealing how attackers exploit open-source models to circumvent built-in security layers.

Exploits, API Attacks, and Security Bypass Tactics

The report also notes a surge in academic and industry research being repurposed for malicious ends, alongside direct attacks on mainstream LLM services like ChatGPT and Gemini.

Prompt injection emerges as a key vulnerability, compelling providers to reinforce defenses with multi-layered guardrails.

Meanwhile, initiatives like DARPA’s AI Cyber Challenge (AIxCC) push for AI-driven solutions to automate vulnerability detection and patching.

Frameworks such as LangChain and the Model Context Protocol (MCP) further enable seamless integration of tools for both analysis and attack automation, including proof-of-concept (PoC) and exploit code generation.

Addressing these threats demands a robust, multi-faceted approach. LLM providers must prioritize defenses against prompt injection and guardrail bypasses while fostering user awareness and leveraging real-time threat intelligence.

Ethical deployment guidelines and community-led response frameworks are critical to balance innovation with security.

The dual-use nature of LLMs-offering immense potential for cybersecurity efficiency while posing significant risks through misuse-requires organizations to adopt proactive, rapid-response strategies.

As generative AI continues to evolve, staying ahead of adversaries exploiting these technologies remains a pressing challenge for the cybersecurity industry, necessitating continuous adaptation to an ever-shifting threat landscape.

Find this News Interesting! Follow us on Google News, LinkedIn, & X to Get Instant Updates!