Stability announces Stable Diffusion 3, a next-generation AI image builder

Stability AI on Thursday announced Stable Diffusion 3, a next-generation image synthesis model with open weights. It follows its predecessors by creating detailed, multi-subject images with improved quality and accuracy in text creation. The brief announcement was not accompanied by a public demo, but stability was Open waiting list Today is for those who want to try it.

Stable says the Stable Diffusion 3 model family (which takes text descriptions called “prompts” and turns them into corresponding images) ranges in size from 800 million to 8 billion parameters. The scale accommodates allowing different versions of the model to run locally on a variety of devices — from smartphones to servers. The parameter size roughly corresponds to the capability of the model in terms of the amount of detail it can generate. Larger models also require more VRAM on the GPU accelerators to run.

Since 2022, we have seen Stable launch its evolution of AI image generation models: Stable Diffusion 1.4, 1.5, 2.0, 2.1, XL, XL Turbo, and now 3. Stability has made a name for itself as providing a more open alternative to proprietary image synthesis models like OpenAI's DALL-E 3, although it is not without controversy due to the use of copyrighted training data. Bias and potential for abuse. (This led to unresolved lawsuits.) The steady-state diffusion models were open-weighted and open-source, meaning that the models could be run locally and tuned to change their outputs.

Regarding the technical improvements, Stability CEO Imad Mushtaq said books On the

As Mostaque said, the Stable family uses Diffusion 3 Structure of diffusion transformersa new method of creating images using artificial intelligence that replaces the usual image building blocks (e.g UNET architecture) for a system that works on small pieces of the image. This method is inspired by transformers, which are good at dealing with patterns and sequences. Not only does this approach increase efficiency, but it is also said to produce higher quality images.

Stable Diffusion 3 is also used”Flow matching“, a technique for creating artificial intelligence models that can create images by learning how to go from random noise to a smoothly structured image. It does this without having to simulate every step of the process, and instead focuses on the general direction or flow that should Image creation follows.

We don't have access to the Stable Diffusion 3 (SD3), but from the samples we found posted on the Stable website and associated social media accounts, the Generations look roughly comparable to other modern photomontage models at the moment. Including the aforementioned DALL-E 3, Adobe Firefly, Imagine with Meta AI, Midjourney, and Google Imagen.

SD3 seems to handle text generation very well in examples provided by others, which are likely cherry-picked. Text generation has been a particular weakness in previous image montages, so improving this ability in freeform is a big deal. Also, the speed accuracy (how closely it follows the descriptions in the prompts) seems similar to DALL-E 3, but we haven't tested that ourselves yet.

While Stable Diffusion 3 is not widely available, Stability says that once testing is complete, its weights will be free to download and run locally. “This preview phase, as with previous models, is critical to gathering ideas to improve its performance and safety before open release,” Stability wrote.

Stability has been experimented with a variety of image montage architectures recently. Apart from the SDXL and SDXL Turbo, the company announced just last week Stable cascadewhich uses a three-stage process to overlay text to an image.

Listing image by Imad Mushtaq (AI for Stability)