Qualcomm Wants To Run AI Tasks Locally On PCs

If you want to use all the beautiful new AI-supported services from Microsoft and Google or Adobe, you always have to use the Internet to do so. Qualcomm wants to change this soon and promises that the first Windows PCs with locally running generative AI can be used later this year.

As part of the Microsoft developer conference Build 2023, Qualcomm will show the native operation of AI models on Windows-based PCs. In the future, users and software providers should also be able to use AI models without an Internet connection and only with the processors in their own computers or mobile device.

Qualcomm chips should offer even more AI performance in the future

As on smartphones, Qualcomm also wants to enable the so-called “On-Device Generative AI” on its platforms for Windows PCs, with which software developers and cloud service providers can carry out certain tasks from the field of generative artificial intelligence on the users’ local devices can relocate. This should make the use of generative AI, i.e. systems for generating texts, music, videos or even images, more affordable, reliable, and private.

So far, Qualcomm has shown, among other things, Stable Diffusion, i.e. an image generator, in a variant running locally on Windows PCs based on a generative AI model with more than a billion parameters. In the coming months, they want to be able to run AI models with up to ten billion parameters locally on their platforms, including Large Language Models (LLMs). In the “next generation”, Qualcomm meanwhile promises significantly more performance, which could probably mean, among other things, the Snapdragon 8cx Gen 4, which for the first time is equipped with the “Oryon” CPU cores from the former Apple chip developers of the startup Nuvia, which Qualcomm took over should be.

In the course of Build 2023, the Qualcomm AI Engine Direct SDK will be made available for the first time, with which you can already process AI workloads on devices with the Qualcomm Snapdragon 8cx Gen 3 with the help of the ONNX Runtime developed by Microsoft and the Qualcomm AI Stack – i.e. on Snapdragon notebooks such as the Surface Pro 9 5G or the Lenovo ThinkPad x13s.

Whether Qualcomm’s announcement means we’ll actually be able to run certain software with built-in AI on ARM PCs soon to have calculations made locally remains to be seen. The prerequisite is, of course, that the software developers accept the chip manufacturer’s “offer” – and so far, the number of applications running natively on ARM PCs with Windows is well known to be very limited.