According to a report on the website of the American Association for the Advancement of Science on the 19th, a team from Princeton University and Stanford University in the United States has developed a new compression algorithm called CALDERA, which can streamline the massive data of large language models (LLMs) and "slim down" LLMs. This algorithm not only helps protect data privacy, save energy, and reduce costs, but also promotes the efficient use of LLM on mobile phones and laptops. The team gave an example that when people use ChatGPT, requests are sent to OpenAI's backend server for processing. This process is not only costly and consumes a lot of energy, but is usually also very slow. If users want to run LLMs using consumer grade graphics processing units, they need to compress these LLMs. The CALDERA algorithm works by reducing LLM redundancy and lowering the accuracy of the information layer. The "slimmed down" LLM is more streamlined, allowing storage and access on devices such as smartphones or laptops, while providing almost the same accurate and subtle performance as the uncompressed version. Although CALDERA is not the first algorithm to compress LLM, its uniqueness lies in combining the characteristics of "low precision" and "low sorting". Among them, 'low precision' reduces the number of bits and speeds up data storage and processing. And 'low ranking' reduces redundancy in LLM data. The team stated that LLM compressed using CALDERA may be suitable for scenarios where accuracy requirements are not the highest. In addition, users can fine tune the compressed LLM on devices such as smartphones or laptops, allowing them to adjust the model according to specific needs to enhance privacy without sharing sensitive data with third parties. However, the team also reminds that running LLM on smartphones or laptops may consume device memory. (New Society)
Edit：Yao jue Responsible editor：Xie Tunan
Source：Science and Technology Daily
Special statement: if the pictures and texts reproduced or quoted on this site infringe your legitimate rights and interests, please contact this site, and this site will correct and delete them in time. For copyright issues and website cooperation, please contact through outlook new era email：lwxsd@liaowanghn.com
Sci-Tech
Recommended Reading Change it 
Home
