IBM
IBM Research recently details disclosed about its NorthPole neural accelerator. This isn’t the first time IBM has discussed this piece; IBM researcher Dr. Dharmendra Modha gave a presentation at Hot Chips last month that delved into some of its technical underpinnings.
Let’s take a high-level look at what IBM announced.
A new type of neural accelerator
IBM NorthPole is an advanced AI chip from IBM Research that integrates processing units and memory on a single chip, significantly improving the energy efficiency and processing speed of artificial intelligence tasks. It is designed for low-precision operations, making it suitable for a wide range of AI applications while eliminating the need for bulky cooling systems.
Architecture of the North Pole
NorthPole is implemented with a new architecture that differs from traditional computer chips, allowing it to perform AI tasks more efficiently. This is how NorthPole works:
- Integrated processing and memory: Unlike traditional chips, NorthPole integrates processing units and memory on the same chip. This integration eliminates the traditional von Neumann bottleneck, where data must shuttle between memory and processing units, resulting in delays and increased power consumption.
- On-chip memory: All memory needed for processing is located directly on the NorthPole chip. This design eliminates the need to access external memory, reducing latency and power consumption. It creates an interleaved network of memory and processing on the chip.
- Efficient inference: NorthPole is designed primarily for AI inference tasks. It excels at processing data quickly and making predictions based on pre-trained AI models. This efficiency is achieved through the integration of specialized memory and processing cores.
- Energetic efficiency: NorthPole is very energy efficient, meaning it can perform many AI operations while consuming relatively little power. This efficiency makes it suitable for use in scenarios where power consumption is an issue, such as edge computing applications.
- Scalability: NorthPole is designed to support many practical AI applications. It can be expanded by breaking larger neural networks into smaller subnetworks that fit into NorthPole’s memory, and multiple NorthPole chips can be connected to handle more complex tasks.
IBM NorthPole Neural Accelerator
NorthPole’s unique architecture, which integrates processing and memory on the same chip and minimizes data transfer between components, results in higher power efficiency, lower latency and improved performance for processing tasks. AI inference. This chip is designed to be efficient, easy to integrate into systems, and suitable for a wide range of AI applications.
Advantages of the North Pole
IBM’s NorthPole has demonstrated exceptional performance in tasks such as image recognition and object detection, outperforming existing chips in terms of performance and efficiency.
In tests with AI systems such as ResNet 50 and Yolo-v4, IBM demonstrated that NorthPole is 25 times more energy efficient and 22 times faster than Nvidia’s V100 GPU. Even compared to more advanced nodes like Nvidia’s H100 GPU, NorthPole is five times more power efficient.
NorthPole’s memory is entirely on-chip, allowing efficient memory access for each core. This architecture also allows NorthPole to appear as an active memory chip from the outside, simplifying integration into new systems.
NorthPole is optimized for low precision operations (2-bit, 4-bit and 8-bit), allowing high precision on neural networks while avoiding the high precision required for training. It operates over a frequency range of 25 to 425 megahertz and can perform 2,048 operations per core per cycle with 8-bit precision. The prototype is built on a 12nm process node.
A notable feature of NorthPole is its ability to efficiently process data without the need for bulky liquid cooling systems, making it suitable for deployment in compact spaces. Ongoing research efforts aim to explore further innovations and advancements in chip processing technologies, promising even greater efficiency and performance gains.
The analyst’s point of view
NorthPole is the culmination of nearly two decades of research at IBM Research focused on creating brain-inspired digital chips. It represents a fusion of traditional processing devices with brain-like processing structures, where memory and processing are closely linked.
The project remained secret until recently, and its success reflects the dedication and collaborative efforts of the IBM Research team. NorthPole marks an important milestone in the quest for energy-efficient computing inspired by the human brain.
NorthPole’s versatility, high power efficiency, and ability to handle low-precision operations make it well-suited for various AI applications, including image analysis, speech recognition, and large language models. Its development opens the door to new innovations in AI hardware.
NorthPole is the latest example of the rapid pace of IBM’s machine learning capabilities, which includes innovations such as the Tellum processor in its latest generation z-series and its impressive cadence of Watson.x developments. At the same time, IBM does not know when the technology demonstrated in the North will be integrated into production hardware; rest assured that this happens.
Disclosure: Steve McDowell is an industry analyst and NAND Research an industrial analytics company that engages or has engaged in research, analysis and consulting services to numerous technology companies, which may include those mentioned in this article. Mr. McDowell has no stock ownership in any of the companies mentioned in this article.