Promptmap – Tool to Test Prompt Injection Attacks on ChatGPT Instances

Prompt injection refers to a technique where users input specific prompts or instructions to influence the responses generated by a language model like ChatGPT.

However, threat actors mainly use this technique to mod the ChatGPT instances for several malicious purposes. It has several negative impacts like:-

• Misinformation
• Content bias
• Offensive content
• Manipulation

An independent security researcher, Utku Sen, recently developed and launched a new tool dubbed “promptmap” that will enable users to test the prompt injection attacks on ChatGPT instances.

Promptmap

On ChatGPT instances, the “promptmap” automatically tests the prompt injections by understanding the context and purpose of your rules configured on ChatGPT.

It uses this understanding to create custom attack prompts for the target, running them alongside your system prompts. While this tool checks for prompt injection success by analyzing the ChatGPT instance’s response.

Attack types

Here below, we have mentioned all the current attack types along with their details:-

• Basic Injection: These attacks are straightforward, as they are sent without prompt enhancements, aiming for unrelated answers or actions.

• Translation Injection: These attacks work by giving English prompts to ChatGPT without language restrictions to gauge if it responds in another language.

• Math Injection: Getting ChatGPT to solve a math equation indicates its capability for complex tasks. However, attacks like math injection prompts can be customized for specific targets.

• Context-Switch: Context-switching involves asking unrelated questions to measure the willingness of ChatGPT to answer sensitive queries that are mainly tailored to specific targets.

• External Browsing: External browsing prompts allow the ChatGPT to browse specific URLs, and they are evolving based on the target’s needs.

• External Prompt Injection: The External Prompt Injection asks ChatGPT if it’s possible for it to access specific URLs for additional prompts.

Installation

Here below we have mentioned the installation procedure:-

• Clone the repository:

git clone https://github.com/utkusen/promptmap.git

• Go inside the folder.

cd promptmap

• Install required libraries

pip3 install -r requirements.txt

• Open promptmap.py file and add your OpenAI API key into the following line: openai.api_key = “YOUR KEY HERE”

You can also change model names that are defined target_model and attack_model variables.

Moreover, with the help of the “python3 promptmap.py command,” the promptmap” can be run, and it defaults to 5 attack prompts per category, which is adjustable with the ‘-n’ parameter.

Keep informed about the latest Cyber Security News by following us on Google News, Linkedin, Twitter, and Facebook.