Hide Your Weaknesses from Attackers: A Defense Method against Black-Box Adversarial Text Attacks

Despite the significant successes of LLMs in generative tasks, the current preference in classification scenarios predominantly leans towards the use of pre-trained language models(PLM), taking into account the balance between cost and effectiveness. However, these models are prone to manipulation through black-box adversarial text attacks, where attackers modify texts in subtle ways to deceive the models. Typically, attackers follow a two-step process: first, they identify crucial sentence elements for the model, then they alter these elements by replacing, deleting, or adding words or characters.
Previous research has mainly focused on creating adversarial examples for training, aiming to improve model resilience. These efforts often overlook defenses against the initial phase of identifying vulnerable targets. This paper introduces a defensive strategy against these first-stage attacks, leveraging concepts from differential privacy. We propose a novel approach, extbf{Mask Regeneration}, which conceals the targets using a [Mask] token and employs a Mask Language Model (MLM) to generate misleading samples. Additionally, we observe that key targets often align with high attention values in the model. Based on this insight, we introduce an extbf{Attention Shuffle} tactic, which randomizes the top-k attention values at each transformer layer, further disorienting attackers.
The experiment shows that our defense method achieves better robustness gains than the State-of-the-art under three strong adversarial attacks for three typical NLP tasks, like sentiment analysis, textual entailment, and topic classification. Moreover, it is also demonstrated that the attack cost significantly increases when attacking our defense model.