Malicious GGUF Models Could Trigger Remote Code Execution on SGLang Servers

Security researchers have uncovered a critical vulnerability in SGLang, a widely used framework for running large language models, that allows threat actors to compromise inference servers.

Tracked as CVE-2026-5760, this flaw enables Remote Code Execution (RCE) when a server loads a maliciously crafted GGUF model file.

By simply hosting a weaponized model on platforms like HuggingFace and tricking a victim into deploying it, attackers can take complete control of the underlying host machine.

The vulnerability stems from a Server-Side Template Injection (SSTI) oversight in SGLang’s reranking endpoint.

When the framework processes chat templates supplied directly by an imported model, it uses an insecure configuration of Jinja2.

Specifically, the vulnerable code path in serving_rerank.py utilizes the standard Jinja2.Environment() function instead of the much safer ImmutableSandboxedEnvironment.

This critical error means templates are rendered without a sandbox, allowing embedded Python commands to execute directly on the server that processes the data.

Attack Execution Steps

The recently published proof-of-concept exploit by Github, demonstrates how an attacker can leverage this unsandboxed environment to achieve code execution:

• The attacker creates a malicious GGUF model file containing a weaponized tokenizer.chat_template.
• The template is embedded with a specific trigger phrase (“The answer can only be ‘yes’ or ‘no'”) designed to activate SGLang’s Qwen3 reranker detection.
• Once an unsuspecting victim downloads and loads the compromised model into their SGLang environment, the trap is set.
• When an API request is made to the /v1/rerank endpoint, the framework routes the request through the vulnerable Jinja2 rendering path.
• The SSTI payload escapes the template context using a known Python bypass technique, successfully executing arbitrary operating system commands on the host server.

This vulnerability highlights a growing supply chain risk within AI infrastructure, specifically concerning how frameworks handle untrusted model metadata.

The flaw falls under CWE-1336 (Improper Neutralization of Special Elements Used in a Template Engine) and CWE-94 (Improper Control of Generation of Code).

It shares the same vulnerability class as the notorious “Llama Drama” bug (CVE-2024-34359) that previously affected the llama-cpp-python library.

Additionally, it shares an attack surface with a recent Denial-of-Service vulnerability in vLLM (CVE-2025-61620).

Administrators running SGLang version 0.5.9 should exercise extreme caution and avoid downloading unverified GGUF models from public repositories until a patch enforcing sandboxed template rendering is applied.

Follow us on Google News, LinkedIn, and X to Get Instant Updates and Set GBH as a Preferred Source in Google.