DeBackdoor: A Framework for Detecting Backdoor Attacks in Deep Learning Models

Deep learning models, increasingly integral to safety-critical systems like self-driving cars and medical devices, are vulnerable to stealthy backdoor attacks.

These attacks involve injecting hidden triggers into models, causing them to misbehave when triggered.

Researchers from the Qatar Computing Research Institute and the Mohamed bin Zayed University of Artificial Intelligence have developed DeBackdoor, a novel framework designed to detect such attacks under realistic constraints.

Addressing Realistic Constraints

In many scenarios, developers obtain deep models from third-party sources without access to the training data or the ability to inspect the model’s internals.

This creates a challenging environment for backdoor detection, as most existing techniques require access to the model’s architecture, training data, or multiple instances of the model.

DeBackdoor addresses these limitations by using a deductive approach to generate candidate triggers and employing a search technique to identify effective triggers.

The framework focuses on optimizing a continuous version of the Attack Success Rate (ASR), a key metric for evaluating backdoor effectiveness.

Detection Methodology

DeBackdoor’s detection methodology involves defining a search space of possible trigger templates based on the description of the attack.

According to the Report, it then uses Simulated Annealing (SA), a stochastic search technique, to iteratively construct and test candidate triggers.

SA is chosen for its ability to avoid local minima, ensuring a more comprehensive exploration of the trigger space compared to simpler methods like Hill Climbing.

By applying these triggers to a small set of clean inputs and evaluating the model’s responses, DeBackdoor can determine if a model is backdoored.

The DeBackdoor framework has demonstrated high detection performance across various attack scenarios, including different trigger types and label strategies such as All2One, All2All, and One2One.

It outperforms existing detection baselines like AEVA and B3D, which are limited in their scope and effectiveness.

The adaptability of DeBackdoor makes it particularly valuable in scenarios where the attack strategy is unknown or diverse, providing a robust solution for ensuring the security of deep learning models in critical applications.

Are you from SOC/DFIR Teams? – Analyse Malware, Phishing Incidents & get live Access with ANY.RUN -> Start Now for Free.