Is the “NPU” the reason for the standard in the AI era?
Apple officially announced the arrival of the “Apple Intelligence” era this year. The future user experience of iPhones, Macs, and iPads will be led by new applications such as AI assistants and machine learning. However, not all Apple users have been “invited” to this party. Only the iPhone 15 Pro and 15 Pro Max models have access, and even the recently launched iPhone 15 is excluded. It’s surprising that a phone released less than a year ago is already considered outdated. In comparison, Mac users only need an Apple Silicon computer launched after 2020. Among all Apple products, the requirements for iPhones seem quite strict.
What restricts the Apple Intelligence feature on iPhones? One might assume that it’s the Neural Processing Unit (NPU) hindering the progress of AI on iPhones. Companies like Microsoft, Intel, and Qualcomm emphasize the importance of having an NPU for generating AI on end-user devices.
What is an NPU? In the past, smartphones and laptops relied mainly on the Central Processing Unit (CPU) and the Graphics Processing Unit (GPU). The CPU excelled at handling complex tasks, while the GPU specialized in graphics processing. However, AI technology requires the capabilities of an NPU. NPUs have numerous small cores, allowing them to efficiently handle repetitive tasks while consuming less energy. This makes them an essential requirement for the next generation of AI PCs and smartphones.
During his speech at the 2024 Taipei International Computer Exhibition, Cristiano Amon, the CEO of Qualcomm, emphasized the importance of NPUs in current AI computing.
But is this the reason why iPhone models below the iPhone 15 Pro cannot use Apple Intelligence? Andrew Williams, a renowned journalist from WIRED, points out that iPhones have been using NPUs since 2017. The first-generation Apple ANE (Apple Neural Engine) was introduced that year and was already an NPU. This means that Apple started incorporating AI features into iOS before AI became a popular term.
The latest iPhone 15 series also includes an NPU. The A16 chip in these models has an NPU performance of 17 TOPS (trillions of operations per second). According to Ming-Chi Kuo, an analyst at TF International Securities, Apple only needs 11 TOPS for AI processing on iPhones, while Microsoft defines the basic requirements for an AI PC as 40 TOPS. The remaining computational power can be obtained by utilizing private cloud computing.
“It’s possible that Apple Intelligence isn’t that different from previous AI, it just has a few additional features,” says Andrew.
So, the real key is not the NPU. The answer may be quite simple: it’s because the RAM memory is not large enough. Ming-Chi Kuo points out that the iPhone 15 series only has 8GB of RAM in the iPhone 15 Pro and iPhone 15 Pro Max models, while the iPhone 15 has only 6GB. This is insufficient to support the computational requirements of Apple Intelligence.
Why is RAM memory important for AI smartphones? When AI models run locally offline instead of being connected to the cloud, they need to be temporarily stored in RAM or vRAM (virtual memory). This requires a significant amount of storage capacity. For example, the NVIDIA H200 used by ChatGPT has 141GB of vRAM per card, and it requires hundreds of such cards to run the service.
Even the highest specification iPhone 15 Pro Max with 8GB of RAM can only handle small and relatively simple AI models. According to Apple’s official website, the offline capabilities include features such as photo review, photo scene recognition, Siri suggestions, voice recognition, and translation. “If you think about it, these offline features don’t include generative AI,” says Andrew. These exciting generative AI applications, such as image generation in chat rooms and document generation in emails, still rely on the cloud.
Ming-Chi Kuo points out that the current Apple Intelligence features require at least a 3B-sized large language model (LLM), and in the future, Apple may upgrade to a 7B model. This will require even more memory to execute. RAM size may become a distinguishing factor for Apple’s high-end and low-end models.
Therefore, there is significant anticipation regarding the configuration of the iPhone 16 and whether it will have the complete Apple Intelligence functionality or will be limited to the Pro series. The answer will be revealed in September this year.
Co-authored article from DIGITIMES.