By sticking a "magic sticker" on your face, you can make the face recognition access control system misjudge and open the door for strangers unprepared; Put this "sticker" on your glasses, and within a second, the mobile phone's face recognition system will be unlocked, and hackers will get users' private data as if they were in an empty land... This is not the imagination in science fiction, but the real attack and defense scene displayed in the first AI security contest. As with other general technologies, in recent years, while AI technology is advancing rapidly, it also brings new risks and hidden dangers. Zhang Bo, an academician of the Chinese Academy of Sciences, said that the development of AI is standing at a new historical starting point. With the availability of computing power, data and other conditions, as well as the progress of machine learning and other technologies, AI has made great progress in computer vision, natural language processing and many other fields, and applications in all walks of life are booming. At the same time, the interpretability, robustness and other defects of the second generation of data driven artificial intelligence have also been exposed, and security incidents have occurred frequently. In real life, the scope of AI technology risk is gradually expanding with the increasingly extensive application scenarios, and the probability of risk occurrence continues to increase with the growth of its application frequency. The demonstration of face recognition cracking reveals the risk of artificial intelligence system, which comes from the vulnerability of deep learning algorithm itself. The second generation of artificial intelligence with deep learning algorithm as its core is a "black box", which is inexplicable, meaning that the system has structural vulnerabilities and may have unpredictable risks. A typical scenario is a "magic sticker", which makes the system make wrong judgments by adding disturbances to the input data. This vulnerability also exists in the automatic driving perception system. Under normal circumstances, after identifying roadblocks, signs, pedestrians and other targets, the autonomous vehicle will stop immediately. However, after adding interference patterns to the target objects, the vehicle's perception system will make mistakes, leading to collision risk. Balancing development and security is an inevitable problem in the development process of each new technology. How to realize the benign interaction between high-level development and high-level security is also an important proposition for the development of the current AI industry. Experts believe that, from the current point of view, paying attention to the construction of AI security system is both an urgent matter and a long-term consideration, and it is necessary to accelerate the research and attack and defense practice of key technologies in the field of AI security. AI counterattack and defense includes counterattack samples, neural network backdoor, model privacy and other technologies. If the model has errors, it needs to be repaired in time. Chen Kai, deputy director of the State Key Laboratory of Information Security of the Chinese Academy of Sciences, proposed the method of "neural network scalpel" to conduct precise "minimally invasive" repair by locating the neurons that cause errors. He said that unlike the traditional model repair work, which requires retraining the model or relies on a large number of data samples, this approach is similar to "minimally invasive surgery", which can greatly improve the model repair effect with very few or no data samples. The AI system in the open environment faces many security challenges. How to solve the security guarantee problem of the whole cycle of the general AI algorithm has become the top priority. Experts suggest that future AI security should focus on data, algorithms and systems