Projects

Project description/goals

This project studies black-box adversarial attacks against deep neural networks (DNNs), where the attacker can only access the query feedback returned by the attacked API, while other information such as model parameters or the training datasets are unknown. In order to protect the security of services which are provided by Tencent, it is necessary to research on black-box adversarial attacks. The proposed attack could be the baseline of security test, and we will design method to defense it.

Importance/impact, challenges/pain points

AI technology has been widely used in Tencent's products and services, such as face recognition, image recognition and other products provided by Tencent Cloud, which provide services to users in the form of API. However, malicious attackers can steal, cheat or leak models’ training data in a black-box attack. Once the attack is achieved, Tencent will suffer from incalculable economic and reputational losses. Therefore, in order to ensure the security of Tencent services, it is necessary to study efficient black-box attacks in real-world scenarios. In addition, the existing defenses have their own shortcomings, such as query, migration, query and migration combination.

Solution description

The solution is to develop an efficient black-box attack for real-world scenarios. First, we could map the simple normal distribution to a complex distribution. Then, we use the efficient training method on surrogate models. Finally, we transfer the mapping parameter to the model trained on surrogate models.

Key contribution/commercial implication

We developed a novel efficient black-box attack based on randomly sampled perturbations.

Next steps

In the next step, we intend to design a defense method to protect models against black-box attacks.

Collaborators/partners

Tencent

Team/contributors

Zeyu Qin    PhD

Mingli Zhu  PhD

Yaopei Zeng   Mphil