When it comes to black boxes, many people associate them with devices used to record flight data on airplanes or nostalgic small theaters. However, in the field of artificial intelligence (AI), black box is also an important term that cannot be ignored. The Spanish newspaper El Pais pointed out that even the most experienced researchers have no knowledge of the internal workings of AI neural networks when they are running. The discussion here is not about biology, but about AI algorithms, especially those based on deep learning that mimic the connections between neurons. These systems are like black boxes, with data scientists, top talents in academia, and Nobel Prize winning engineers from OpenAI and Google finding it difficult to uncover their internal mysteries. The model and data are opaque Scientific American magazine reported that AI black box refers to the AI system whose internal operation mode is completely invisible to users. Users can input information into these systems and obtain output, but they cannot check their code or understand the logic that generates the output. Machine learning, as the main branch of AI, is the cornerstone of generative AI systems such as ChatGPT. Machine learning consists of three core components: algorithms, training data, and models. An algorithm is a series of program instructions that, in machine learning, learn to recognize patterns in data through a large amount of training data. When machine learning algorithms complete training, the product is the machine learning model, which is also the part that users actually use. Any of these three parts of a machine learning system may be hidden, i.e. placed in a black box. Usually, algorithms are publicly available. But in order to protect intellectual property, AI software developers often put models or training data into black boxes. The model architecture is so complex that it is difficult to explain. Although the mathematical principles behind many AI algorithms have been fully understood, the behavior generated by the networks composed of these algorithms is elusive. ChatGPT, Gemini, Claude, Llama, and any image generator like DALL-E, as well as any system that relies on neural networks, including facial recognition applications and content recommendation engines, all face this problem. In contrast, other AI algorithms such as decision trees or linear regression (commonly used in fields such as medicine and economics) are more interpretable. Their decision-making process is easy to understand and visualize. Engineers can follow the branches of the decision tree to clearly see how specific results are obtained. This clarity is crucial as it injects transparency into AI and provides security for algorithm users. It is worth noting that the EU's Artificial Intelligence Act emphasizes the importance of having transparent and interpretable systems. However, the architecture of neural networks themselves hinders this transparency. To understand the black box problem of these algorithms, one must imagine a network composed of interconnected neurons or nodes. Juan Antonio, a professor at the AI Institute of the Spanish National Research Council, explained that when you input data into a network, the values in the nodes trigger a series of calculations. The information starts to propagate from the first batch of nodes and is transmitted in numerical form to subsequent nodes. Each node calculates a number and sends it to all connections, while considering the weight (i.e. numerical value) of each connection. The new node that receives this information will calculate another number. It is worth noting that current deep learning models contain thousands to millions of parameters. These parameters represent the number of nodes and connections after training, which is vast and varied, making it difficult to manually derive meaningful equations. According to industry insiders, GPT-4 has nearly 1.8 trillion parameters. According to this analysis, each language model will use approximately 220 billion parameters. This means that every time a question is posed, there are 220 billion variables that may affect the algorithm's response. The attempt by technology companies to unlock the opacity of black box systems has made it more difficult to correct biases and has exacerbated feelings of distrust. At present, the main participants in the AI field have realized this limitation and are actively conducting research to better understand the working principles of their models. For example, OpenAI uses neural networks to observe and analyze another neural network, while Anthropic studies node connections and information propagation circuits. Decoding black boxes is of great benefit to language models, as it can avoid erroneous reasoning and misleading information generated by AI, and solve the problem of inconsistent answers. However, due to a lack of understanding of the internal mechanisms of the network, technology companies often train their models extensively and release their products after testing. This method may also have issues, such as Google Gemini generating incorrect images upon initial release. The opposite concept to a black box is a glass box. The AI glass box refers to its algorithms, training data, and models that can be seen by anyone. The ultimate goal of decoding black boxes is to maintain control over AI, especially when deployed in sensitive areas. Assuming a machine learning model has already made a diagnosis of human health or financial condition, would people prefer the model to be a black box or a glass box? The answer is obvious. This is not only a high level of attention to the internal working principles of the algorithm, but also out of scientific curiosity and the protection of user privacy. (New Society)