Recently, the 30thACM International Conference on Multimedia (ACM MM 2022), a top international conference in computer graphics and multimedia, was held in Lisbon, Portugal. It is listed as a Class A conference by China Computer Federation (CCF). The latest research findings "Purifier: Plug-and-play Backdoor Mitigation for Pre-trained Models Via Anomaly Activation Suppression" by Professor Chen Xiaofeng's team from the School of Network and Information Security was included in the conference proceedings and the team was invited to make a report at the conference.
The first author of the research findings is Associate Professor Zhang Xiaoyu from the Xidian University Ruiyun Data Security Team, who is the winner of the TRS Excellent Doctoral Dissertation Award by the Chinese Information Processing Society of China in 2020.
This paper focuses on the backdoor defense in deep learning pre-training models. Backdoor attacks usually occur in non-fully controlled training process scenarios, posing potential threats to the security of prediction models. In response to this, this paper proposes Purifier, a backdoor defense method based on anomaly activation suppression. As shown by the feature representations of the middle layer of the model, the visually contrasting performance of the pre-training model in the face of backdoor samples and clean samples intuitively reveals an essential problem: backdoor samples would present anomaly schema in the representation of the middle layer. Furthermore, by dynamically optimizing and updating the weight value corresponding to the fine-grained unit, the researcher can suppress the effect of abnormal activation of neurons, so as to resist multiple unknown types of backdoor attacks. Compared with the existing SOTA backdoor defense methods, Purifier does not need to access the original pre-training data and only requires a small amount (about 1%) of downstream task clean data. The experimental results with six mainstream attacks show that Purifier has the advantages of high efficiency, strong applicability and interpretability. In addition, Purifier has a plug-and-play feature, which is flexible enough for the pre-training model of any network structure, and has solved some common problems with the algorithm for backdoor defense, such as high complexity and inflexible applicability.
Purifier Framework
In recent years, the School of Network and Information Security has vigorously stepped into the cross fields of security, encouraging teachers and students to actively explore emerging research areas driven by their initial interest. The findings are one of the key achievements made by the Ruiyun Data Security Team in artificial intelligence (AI) security, which will further drive the overall development of scientific research in the school.
