AI-powered art generators are fun and increasingly popular tools. They allow images and artwork to be created by typing a prompt and letting the AI model, which will have been trained on a dataset of billions of images, do its thing. OpenAI’s DALL-E 2 is the primary example, but there are several others, including Stability AI’s Stable Diffusion.
Stable Diffusion is an open-source large-scale artificial intelligence model, trained on a subset of 2.3 billion English-tagged images from the LAION 5B dataset, which contains 5.85 billion pairs image-text. You can access Stable Diffusion in cloud mode via the web, or run it on a PC with a GPU that has 6GB or more of video RAM, or on an Apple with Apple Silicon running MacOS (13.1) or iOS (16.2 ). The latest version of Stable Diffusion, 2.1, was released in December 2022.
At Mobile World Congress (MWC), Qualcomm is showing for the first time a Stable Diffusion instance running locally on a high-end smartphone equipped with the company’s latest Snapdragon 8 Gen 2 chip.
Picture: Qualcomm
This development is the work of the Qualcomm AI Research team, which leverages AI Stack and Qualcomm’s developer tools. The Stable Diffusion language model is quantized and optimized using int8, allowing the entire model to be stored on the smartphone. Four AI stack optimizations generate a high-quality 512 by 512 pixel image in less than 15 seconds from query input, showing 20 inference steps from random pixels to finished image . “We are continually improving latency and accuracy,” said Mike Roberts, vice president of Qualcomm.
Performance, privacy, security and cost
The move toward processing AI workloads on mobile devices rather than in the cloud is driven by performance, privacy, security, and cost considerations. “The more there is on the device, the more we can scale this technology to applications,” Roberts said.
“We really want to show that this is how we can evolve devices and models to make AI truly ubiquitous,” he added. Here are some examples of what stable streaming can do (on the web):
Image: Stable Diffusion
Image: Stable Diffusion
One caveat: the huge databases used to train AI art generators scrape images from the internet and may include inappropriate or copyrighted material. Getty Images is currently suing Stable AI for using 12 million of its images from the LAION 5B dataset without permission to train Stable Diffusion.