Integrating storage and computing into one is different from the von Neumann architecture design that computers have been following for the past 70 years. As the world enters the era of artificial intelligence, the demand for high bandwidth and low power consumption in computing scenarios is increasing, and the industry's attention to the concept of "compute-in-memory" (CIM) has also risen. Recently, a research output from Alibaba DAMO Academy has pushed this concept back into the limelight: Alibaba Cloud revealed in a statement that DAMO Academy has successfully developed a compute-in-memory AI chip. Alibaba Cloud stated that this is the world's first 3D bonded stacking compute-in-memory chip based on DRAM.

Bringing It Closer – Coexistence – Integration: The Compute-in-Memory Chip Is Developing Along This Technological Path.
Breaking Down the "Wall" Between Storage and Computing
Under the von Neumann architecture, data must be transferred through a data bus from storage to processing. Compute-in-memory (CIM) chips integrate storage and computing, directly using storage units for computation, which can effectively address the "storage wall bottleneck" and "power wall bottleneck" caused by data transfer from memory to the processor. "Compute-in-memory technology is a disruption to the traditional von Neumann architecture and is the future trend," said Zhong Xinlong, a senior consultant at the Informationization and Software Industry Research Institute of CCID Consulting, in an interview with China Electronics News.
The concept of compute-in-memory can be traced back to the 1960s, but it has remained lukewarm in the industry due to complexities in chip design, manufacturing costs, and lack of demand drivers. In the past five years, with the rise of artificial intelligence technology globally, the advantages of compute-in-memory in terms of bandwidth and power consumption are considered key to meeting specific AI scenario demands. Both domestically and internationally, compute-in-memory is still in its early stages, and this architecture is in a critical period of transitioning from academia to industry. Companies such as ZhiCun Technology, Pingxin Technology, Jiutian RuiChip, Houmo Intelligence, Hefei Hengshuo, Shanyi Technology, Xinyi Technology, and Hangzhou ZhiXinKe are diligently researching compute-in-memory AI chips. Internationally, Samsung and Mythic are also significant players in this field.
The "Three-Step Strategy" for Compute-in-Memory
The development of a generalized compute-storage integration architecture is not achieved overnight. Alibaba DAMO Academy believes that compute-in-memory is following a "three-step strategy."
The first step, the near-term strategy, is crucial for reducing the distance between storage and computing units, increasing bandwidth, reducing the cost of data movement, and alleviating the bottleneck caused by data movement through chip design, integration, and packaging. DAMO Academy refers to this as "near-storage chips," which is the 3D bonded stacking compute-in-memory chip based on DRAM that has been developed.
The second step, the medium-term plan, involves architectural innovation to either place memory within computing units or computing units within memory modules, achieving a true integration where computing and storage are interdependent. The third step, the long-term vision, is to innovate at the device level so that the device is both a storage unit and a computing unit, without distinction, fully integrated into a true compute-storage integration.
"Transitioning from experimental products to large-scale application and popularization in the consumer and enterprise markets may require ten years or even longer for technological iteration and optimization, continuously improving," Zhong Xinlong said to the reporter.
Xu Chang, a senior analyst at the Artificial Intelligence Industry Research Center of CCID Consulting, pointed out that compute-in-memory chips focus on the edge computing field, which requires low latency, low power consumption, and high computing power, and are suitable for deep learning neural network fields, especially for battery-powered IoT terminal devices such as smartphones, wearable devices, smart homes, drones, smart cameras, and hearing aids.
Zhong Xinlong believes that compute-in-memory chips have a natural advantage in massive data computing scenarios and have broad application prospects in terminals, edge computing, and the cloud. For example, in scenarios such as VR/AR, autonomous driving, astronomical data computing, and remote sensing image data analysis, compute-in-memory chips can leverage their high bandwidth and low power consumption advantages. In the future, compute-in-memory technology will also become a key enabler for brain-like computing.