On Tuesday, the international authoritative journal Nature published a new achievement of AI (Artificial Intelligence). It is an AI image detection software for paper review. It can help editors review fake images, misplaced pictures, etc. in biological papers. In scientific papers, the accuracy of experimental results often needs experimental data to prove. In order to quickly complete the paper or verify their own experimental results, some researchers may use computers to generate false pictures, fabricate data, or use old data to duplicate the map. How to find the modified and copied images in papers faster and more accurately has always been a top priority for academic journal editors. Previously, in ten journals published by the American Association for cancer research (AACR), papers had to undergo an unusual additional examination before publication - reviewing the images in the articles. Since January 2021, AACR began to use artificial intelligence software to re-examine other peer-reviewed manuscripts. In addition to repeating pictures, the software can also find those false images that have been rotated, stretched or generated and modified by computer. ▲ an example of digital color transmission electron microscope (TEM) operation on virus particle images in proofig. The blue line represents hundreds of identical features used by artificial intelligence for comparison 1、 Low cost and good effect, still need to be checked manually Artificial Intelligence Review software will be the development trend in the future, and AACR is the early adopter of this technology. Whether intentionally or unintentionally, in order to avoid publishing tampered images in papers, many journals hire personnel to manually scan manuscripts for problems, and then use review software to help check what they find. However, nature learned that in the past year, several publishers began to automate the review process and rely on artificial intelligence software to find copied and modified images before the manuscript was published. ▲ in the flow cytometry diagram in the paper, there is a completely consistent final cell landing point. (picture source: China Youth Network) Daniel Evanko, director of journal operations of the Philadelphia Association in Pennsylvania, said that AACR tried a variety of software products before finally choosing the service of proofig, an artificial intelligence detection software of Rehovot company in Israel. "We are very satisfied with this." He added that he hoped artificial intelligence could help researchers review to reduce problems after publication. It is worth noting that when AI software marks images, professional editors are still needed to decide what to do. For example, if the same data set appears twice in the text but represents different meanings, the repeated images need to be retained. Secondly, due to the simple copy and paste errors in the process of manuscript assembly, the problem of image duplication will also occur, but this occurs unconsciously and does not deliberately deceive readers. Before that, these problems can only be solved through discussion between editors and authors. In addition, artificial intelligence has become sufficiently effective and low-cost. Experts say that automatic image inspection assistant may sweep the scientific publishing industry in the next few years, just as using software to check plagiarism has become a routine in a decade. Publishing groups also say they are exploring ways to compare images in manuscripts from different journals. Other image integrity experts also recognize this development trend, but they believe that there is no public comparison of various software products, and automatic inspection may cause too many false positives or omit some pictures. In the long run, the dependence on software filtering may also encourage fraudsters to use artificial intelligence to cheat software, just as some people avoid plagiarism filtering by adjusting text. Bernd pulverer, editor in chief of the German Heidelberg EBO reports journal, said: "I am worried that we are in an arms race with artificial intelligence based technology, which may lead to the emergence of deeply forged images." 2、 It is not only used for repeated detection, but also improves picture quality Researchers have been developing artificial intelligence software for image inspection for many years because they are worried that academic misconduct in the paper may cause damage to the scientific literature, which will far exceed the damage caused by withdrawing the article or correcting the statement after the problem in the article. Previously, Elisabeth bik, a Dutch microbiologist, published about 20000 biomedical papers independently or jointly. In 2016, after researchers manually analyzed the pictures in these papers, the results showed that more than 4% of the papers may contain problematic pictures. However, usually only about 1% of the papers are corrected every year, and fewer are withdrawn. ▲ Elisabeth bik, a Dutch microbiologist "I know about 20 companies around the world are developing image inspection software." Said Mike rossner, President of image data integrity, a biomedical research and image processing consulting company in California. Last year, some science publishers around the world jointly set up a working group to set standards for software for screening picture problems; This year, the group issued guidelines on how editors should deal with tampered images, but has not yet developed software guidelines. Some academic groups and companies told nature that journals and government agencies were trying out their AI software, but proofig was the first company to disclose customer information. In addition to AACR, The American Society of clinical investigation began using proofig's software to review manuscripts in the Journal of clinical investigation (JCI) in July, said Sarah Jackson, executive editor of the Journal of the American Society of clinical investigation. Helen king, head of transformation at Sage, an independent academic publishing company in London Helen king said that sage publishing magazine adopted the software in its five life science journals in October this year. Proofig's software extracts images from the paper and compares them in pairs to find common features. Generally speaking, the examination time of papers is about a minute or two. Dror kolodkin gal, the founder of proofig, said that the software can also correct thorny problems, such as compression artifacts that may occur when compressing high-resolution raw data into smaller files. "Computers have more advantages than human vision." "Not only will the computer not feel tired, it will run faster, but it will not be affected by size, position, direction, overlap, partial replication and the combination of these factors," he said With regard to the cost of image inspection, gale declined to discuss the pricing in detail, but said that contracts signed with publishers often charge according to the number of images in the paper, but also depends on the number of manuscripts. He said that this is equivalent to the charge per piece of paper "close to tens of dollars rather than hundreds of dollars". Jackson said that in the Journal of clinical investigation, the software found more problems than previously manually reviewed by staff. Although the staff still need to check the output of proofig, it is important that the journal already has an artificial intelligence system to deal with various image problems. "We really think rigorous data is the absolute symbol of our journal. We think it's worth spending time and money." Jackson said. At the same time, in the Journal of the American Association for cancer research, evanke said that many authors were happy to find their picture duplication errors before publication. 3、 AI + manual, self-developed software by multiple Publishers meanwhile, Frontiers, a publisher based in Lausanne, Switzerland, has developed its own image inspection software Aira (Artificial Intelligence Review assistant) as part of the automatic inspection system. A spokesman said that since August 2020, a team studying image integrity has been using Aira internally to conduct image inspection on all submitted manuscripts. However, most of the papers marked by the software actually have no problems, and only about 10% need to be followed up by the editing team. Frontiers Declined to disclose the number of papers marked by Aira. Image integrity experts, including bick and Rosner, said that they had not tried Aira or proofig, and it was difficult to evaluate software products that had not been publicly compared with standardized tests. In addition to duplication, it is also important to detect whether parts of the image are deleted or cropped through PS, rosener added. "Ai software may be a useful supplement to visual screening, but it may not replace the current form." "However, I believe this will eventually become the standard for manuscript screening." Bick added. Publishers that have not yet adopted AI software for image screening have raised issues of cost and reliability. A spokesman for PLoS said he was "eager" to monitor the progress of the research. These tools can "reliably identify common image integrity problems and can be applied on a large scale". Elsevier, the Dutch publishing group, said they were "still testing" the software, but some of its journals screened all papers before publication and checked the images "using a combination of software tools and manual analysis". A spokesman said that in April 2020, Wiley, an academic publisher, launched an image screening service for temporarily accepted manuscripts. At present, it is used by more than 120 journals, but at present, it is a manual screening assisted by software. Springer nature, publisher of Nature magazine, said it was evaluating some external tools and collating data to train its software, which will "combine complementary artificial intelligence and artificial intelligence to identify problematic images". 4、 Western blotting can be forged to promote cross publisher detection "EMBO press magazine still mainly uses manual screening, and I don't believe in the cost-benefit ratio of commercial products," purwell said And pulwell is a member of the cross publisher working group that defines software standards. "I have no doubt that we will have advanced tools soon." Purwell worries that fraudsters may understand how the software works and use artificial intelligence software to generate false images that neither people nor software can detect. Although no one has indicated that such images have appeared in research papers, a preprint published on biorxiv, a website providing distribution services, last year shows that there may be forged versions of biological images that cannot be distinguished from real data, such as protein prints. ▲ Western blot pictures in papers in the Journal of cell biochemistry (picture source: China Youth Network) But researchers are solving this problem, Edward DELP, a computer scientist at Purdue University in West Lafayette, Indiana (Edward DELP) led a team in a project funded by the Defense Advanced Research Projects Agency. The team is studying software for detecting artificial intelligence forged images and focusing on forged biological images, such as microscope images and X-rays. A paper describing the system is under review. ▲ optical microscope captures micrographs of cells and tissues At present, artificial intelligence image inspection is usually completed in manuscripts rather than in papers, which will increase the amount of calculation. But commercial and academic software developers say this is technically feasible. Daniel Acuna, a computer scientist at Syracuse University in New York (Daniel Acu a), spent thousands of COVID-19 in the past year.