How to Make Stable Diffusion Faster in 5 Ways

 “I recently put together a new PC with a fairly decent setup, however, Stable Diffusion takes over 1 minute to generate a single 512x768 image and if I ask for 4 images, I get Out of Memory errors. How to make Stable Diffusion faster?”

“Stable Diffusion is too slow on my computer. To upscale an image by x2, I'm getting in approx. 8min to 10min or even more than that. Ways to speed up Stable Diffusion Automatic1111?"

Stable Diffusion is taking too long to generate or upscale an image? It’s really frustrating if it takes minutes to generate images that you need. Fear not, creative minds! Here we share different ways to dramatically speed up Stable Diffusion workflow. By implementing these strategies, you'll transform Stable Diffusion from a sluggish tool into a lightning-fast engine for your creative exploration. Get ready to unleash your artistic potential and generate stunning images in a fraction of the time.

1. Optimize Your Hardware

Stable Diffusion relies heavily on your graphics processing unit (GPU) for processing. Generating a single batch of four images takes minutes on nonspecialized hardware. For example, running Stable Diffusion on an AMD GPU is a nightmare. The speed is also slow on CPUs. To get Stable Diffusiion to run faster, you may use a modern, powerful GPU with a significant amount of memory (ideally 8GB or more), such as GeForce RTX 4090. Consider upgrading your GPU if feasible, but even optimizing settings within your existing system can make a difference.

2. Token Merging

One powerful technique to make Stable Diffusion faster is token merging. This involves combining similar or redundant words within your prompt and negative prompt, reducing the number of tokens the model needs to process. This translates to faster generation times, but there's a catch: excessive merging can compromise the final image quality.

Finding the Sweet Spot:

Tools like AUTOMATIC1111 offer built-in token merging functionality. The amount of token merging is controlled by the percentage of token merged. For example, 0.2 would mean that it is merging 20% of the tokens. Navigate to Settings > Optimizations and adjust the token ratio between 0.2 and 0.5. Remember, the higher the ratio, the faster the generation, but also the greater the potential loss of detail. Values above 0.6 can significantly impact image quality.

3. Cross-attention Optimization

Another effective way to speed up Stable Diffusion is through cross-attention optimization. This technique accelerates Stable Diffusion's cross-attention calculations, significantly reducing processing time without demanding exorbitant memory resources. The optimal cross-attention optimization approach depends heavily on the software you're using with Stable Diffusion. 

Optimizing Within AUTOMATIC1111:

1. Access Settings: Navigate to the Settings page within the AUTOMATIC1111 interface.
2. Seek Optimization: Locate the Optimization tab within the settings menu.
3. Fine-Tune Cross-Attention: Look for the Cross Attention Optimization feature. It will likely display a dropdown menu.
4. Automatic Selection (Recommended): The default setting is typically Automatic. This allows the software to choose the most suitable optimization option for your system. However, you can explore other options if desired.
5. Apply Changes: Once you've selected your preferred optimization method, click Apply Settings to activate the changes.

AUTOMATIC1111 offers a diverse toolbox of cross-attention optimization techniques beyond the automatic selection. Here are some prominent options to make Stable Diffusion run faster:

• Doggettx: A well-established method for speeding up cross-attention calculations.
• xFormers: Another powerful option focused on acceleration.
• Sub-quadratic Attention: This technique prioritizes reducing memory consumption during cross-attention calculations.

4. Negative Guidance Minimum Sigma

Negative guidance minimum sigma offers a more nuanced approach to accelerating Stable Diffusion. This technique strategically ignores minor details in your negative prompts, potentially reducing generation time without compromising the final image significantly. Negative prompts tell the model what not to generate. By selectively ignoring minor details, you can streamline the generation process. However, the speed improvements are typically subtle, so don't expect dramatic reductions in generation time.

Using negative guidance minimum sigma:

• Navigate to Settings: Locate the Settings page within AUTOMATIC1111.
• Seek Optimization: Look for the Optimization tab.
• Fine-Tune Negative Guidance: Find the option labeled Negative Guidance Minimum Sigma.
• Setting the Value: Adjust the value to your preference. A higher value translates to ignoring more minor details in your negative prompts, potentially leading to faster generation. However, this also increases the risk of unintended consequences in the final image.

Compared to token merging, negative guidance minimum sigma achieves comparable speed increases with less alteration to the generated images.

5. Set a Lower Resolution or Use A Faster Upscaler

Stable Diffusion is a powerful tool, but upscaling images, especially from 512p to high resolutions, can be computationally expensive and time-consuming. Additionally, optimizing settings, like cross-attention and token merging, to speed up Stable Diffusion can be complex for some users. You can try to upscale Stable Diffusion images to a lower resolution to get a faster speed.

If you'd rather spend more time on the creative aspects of generating images, consider alternative upscalers specifically designed for AI-generated images. Try alternative upscaler like Aiarty Image Enhancer– an AI upscaler specially designed for AI images from Stable Diffusion, Midjourney, and other platforms. It’s well designed for GPU/CPU from Nivida, Intel, and AMD for the fastest speed. Easily upscale images to 4K, 8K, 16K, and higher with natural details. Denoise, deblur, deJPEG, and upscale in just one click.

Conclusion

Conquer Stable Diffusion's slowdowns! Upgrade your GPU and optimize settings for a performance boost. Craft concise prompts, strategically use negative guidance, and explore token merging (with caution) to streamline generation. For upscaling specifically, consider AI upscalers designed for speed and compatibility with AI-generated images. With these tactics, you'll be generating stunning images at breakneck speed, unleashing your creative potential.