Learn how to upscale Stable Diffusion art with ControlNet Upscale. Ever tried to transform your favorite small image into a massive printable billboard, only to be met with unnatural smoothness, loss of details, or quirky artifacts? If you’ve tried tools like Photoshop or GigaPixel, you know the results can be hit-or-miss, with the “hits” often still lacking in interpolated details. We’ve all felt that frustration. In this guide, we will learn how to upscale using ControlNet.nnThe trick to pristine upscaling lies in a tech duo: ControlNet’s Tile model combined with Ultimate SD Upscale. This power combo certifies your upscaled images don’t just get bigger but get brilliantly better with new realistic and natural looking details.

I’ve scoured expert opinions, gathered data, and boiled it all down to give you the clearest, most practical advice on using these tools. .

Upscaling to any desired resolution is a breeze with ControlNet’s Tile. At its core, this tool segments an image into manageable tiles, optimizing each for precision before merging them into a high-resolution masterpiece. While ControlNet Tile lays the foundation for clear and detailed upscaling, introducing the power of Stable Diffusion and the Ultimate SD Upscaler further elevates the quality.

Wondering exactly how ControlNet Tile transforms your images? Or curious about the mechanics behind Stable Diffusion? Keep reading the following sections to understand the process and learn how these tools synergize to create unparalleled upscaling results.

Ditch those fuzzy results; Learn how AI can make your photos pop in high resolution!

---

What Does ControlNet Tile do?

At its core, ControlNet Tile operates like an intelligent tiling system. Rather than arbitrarily enlarging your image, it carefully assesses and accurately replicates every tiny section in 512×512 chunks. By working on these manageable pieces, it can pay attention to the intricacies of each segment. When these sections are finally pieced together, the end result is a larger image that hasn’t lost any detail. In fact, with the additional power of text-driven guidance, it often introduces new, harmonious details. So, instead of a simple upscale, what you get is an image that not only retains but often surpasses the original’s clarity and vibrance. I’ve tried to incorporate a clearer understanding of the process while maintaining a reader-friendly tone.

Upscale in Automatic1111 Using ControlNet Upscale?

Upscaling in Automatic1111 is akin to tailoring a custom suit – there are several tools and techniques at your disposal. Within A1111, you have a roster of built-in upscaler tools and the Hires.fix method to boost resolution. But the catch: many of these tools demand a GPU that packs a punch. If you’re wielding a GTX 1080 card, some methods could have your PC gasping for breath. With the ControlNet Tiles preprocessor, categorized as a Control Type, those with modest PCs are no longer left behind, yet it’s still a gem for the powerhouse Stable Diffusion users.

When paired with the Ultimate SD Upscaler, this dynamic duo ensures expansive, intricately detailed images, sidestepping the common distortions or oddities that sometimes pop up in aggressive upscaling. Bid farewell to unexpected third limbs and mirrored oddities, ushering in a refined and detail-rich upscaling process.

• How to Install Stable Diffusion and ControlNet:

  We’ve previously discussed how to install Stable Diffusion and ControlNet, for specifics of installing these tools. Go to the following links below:

  • Prerequisites Installation Guide to Stable Diffusion’s Web UI
  • How to Install Automatic1111 Web UI for Stable Diffusion
  • How to Install ControlNet Automatic1111: A Comprehensive Guide

• How to Install Ultimate SD Upscale:

  • Navigate to Extensions in A1111 Stable Diffusion Web UI.
  • Click on Load, then the Available Tab.
  • Select Load From.
  • Use the search bar to find “Ultimate SD Upscaler“.
  • Click Install under Action.
  • Move to the Install Tab, hit Apply and then restart UI.

• For optimal upscaling, the 4x UltraSharp installation is recommended.

  If you haven’t already installed it, here’s what you need to do:

  Download 4x UltraSharp

  • Once downloaded, locate the files.
  • Installation: Move the files to your A1111 installation path, specifically:
    A1111 Installation > Models > ESRGAN Folder.

• Optional and Organized method: Installing Extensions from GitHub Desktop App

  • If you’re keen on a hands-on approach, GitHub Desktop offers an alternative installation method. Explore the detailed steps here or directly head to the repository here.

---

To upscale images using Stable Diffusion, it’s essential to navigate to the “img2img2” setting, where the “Ultimate SD upscale” extension is accessible from the “Scripts” pull-down menu. If it’s not immediately visible, a quick restart of Stable Diffusion often rectifies this.

A pivotal setting is the Control Mode, especially the “ControlNet is more important” option, which significantly influences the image’s final appearance. It’s also crucial to adjust the “Target size type” to “Scale to Image Size”, which bypasses specific parameters like Width and Height.

The desired scale usually ranges between 2 and 4, with the upscaler set to the 4x-UltraSharp Model for optimal results. While several settings like Type and Seams Fix can be tailored according to personal preference, it’s noteworthy that using ControlNet Tile maintains much of the original image’s integrity.

Adjusting the denoising strength and the scale can modify the details and render time, offering users a versatile upscaling experience. Follow the steps below for a streamlined process on how to use it.

How to Set Up ControlNet Tile Preprocessor

1. Mode Selection:

• Switch to “img2img” mode (this process is not applicable in “txt2img”).

2. Go to the “ControlNet” dropdown:

• Upload your source image to upscale.

• Click “Enable”. Ensure this is active to avoid anomalies like visible grids.

• In Control Type, click “Tile” (This should select the “tile_resample” preprocessor.

• In Control Mode, click “ControlNet is more important” (This is important, but play around around with it to see how it changes your images.)

• In Resize Mode, click the one that most applies to you. It is based on the size of your image, so play around with it.

Go to the “Scripts” dropdown.

• Choose “Ultimate SD upscale”.

  (If it’s missing, restart Stable Diffusion.)

• In Target size type:

  select “Scale from image size” Selecting “Scale to Image Size” bypasses the preceding parameters specific to “img2img2”, such as Width, Height, and other related adjustments.

• Scale:

  Ideally, pick a value between 2-4. More if you plan on doing something massive, but I have a better trick than increasing the Scale below.
  Each number represents a multiplier: 1 corresponds to x1, 2 to x2, and so on.

• In Upscaler:

  Click “4x-UltraSharp.”

• Type: Set to “Linear”.

  You can later experiment with other types like “chest”.

• Seam Adjustments:

  Opt for “none” to avoid visible seams.

Upscale your Images

• Ensure all settings are properly configured as previously discussed.

• Navigate to the img2img tab.

• Drag and drop or upload the desired image you wish to upscale.

• Make any final adjustments if necessary.

• Click on the Generate button to begin the upscaling process.

  

To provide a clearer understanding of each feature’s impact, we’ve conducted split tests using the methods previously discussed. Here, you can observe the results when each feature is applied individually, as well as when they are combined. This will help you gauge the effectiveness of each technique and decide on the optimal approach for your upscaling needs.

Ultimate SD Upscale + Denoising Strength

In a resolution test using only the Ultimate SD Upscaler without ControlNet, distinct deformations appeared throughout the image. This is attributed to the upscaler’s method of processing in tiles, rendering separate image sections which can sometimes misalign due to each section’s independent generation. Nevertheless, the upscaling’s clarity is commendable. To discern the nuanced differences between resolutions of 1024×1024 and 2048×2048, a closer inspection is required, but the heightened sharpness is unmistakable. For these tests, scales of 2 and 4 were applied for the 1024 and 2048 resolutions, respectively.

Manual Image Resize and Denoising Strength

In the subsequent experiment, both ControlNet and Ultimate SD Upscaler were deactivated. Instead, the image dimensions were manually adjusted to 1024×1024, with denoising strengths set at 0.2 and 0.5 for evaluation.

At the 0.2 denoising strength, the image, though technically upscaled, seemingly retained its original appearance, exhibiting pixelation characteristic of its initial low resolution. This is attributed to the conservative nature of the 0.2 denoising strength, which closely replicates the source’s attributes, preserving the inherent low resolution. Conversely, at a denoising strength of 0.5, the pixelation diminishes, yielding a more refined, albeit softer, image. However, this technique compromises the subject’s authenticity, rendering it markedly different from the original. While this method might produce aesthetically pleasing results, it doesn’t suffice for genuine upscaling that preserves the original’s essence. Although the softer touch can be appealing for certain images, it’s not the desired outcome for this specific assessment.

ControlNet Tile Preprocessor + Ultimate SD Upscaler + 4x UltraSharp

Leveraging the combined strength of ControlNet and the Tiling preprocessor provides users with enhanced control over the upscaling process using the Ultimate SD Upscaler. Notably, even with increased denoising, the changes aren’t overly dramatic, but for optimal results, it’s advisable to maintain denoising at approximately 0.2. This synergy of techniques produces an upscale quality previously unattainable with conventional tools, showcasing remarkable enhancement in image resolution and detail.

Utilizing the stepping method with ControlNet and the Ultimate Upscaler involves a layered approach to upscaling. In this technique, once an image is upscaled, it is then reintroduced to the upscaler for another round of enhancement. We aim to contrast this method against a direct upscale approach, where the scale is simply set to 6. By comparing a base image of 512×512 against its upscaled 4080 version using both methods, we aim to discern which technique offers superior image quality and clarity, assessing if the iterative stepping method offers any distinct advantage over a straightforward scale increase.n

Pay attention to the intricacies – the fine details of the eyelashes, the textural richness in the cheeks, and the authentic appearance of the eyes. When using a scale level of 8, some unnatural sharpening emerges. However, by employing the regeneration approach detailed below, one can achieve a more refined result when utilizing ControlNet, Tile Preprocessor, Ultimate SD Upscaler, and 4x-UltraSharp together.

• Initial Setup:

  Begin with your source image and ensure the scale is set to 2 within your upscaling software.

• First Upscale:

  Generate your upscaled image using your desired settings.

• Reintroduce the Upscaled Image:

  Once your image is upscaled, take the resulting generated image (which will typically appear on the right side of your software interface) and drag it to the left side. This effectively makes the newly upscaled image your new source. Alternatively, you can save the upscaled image and upload it again as the source.

• Repeat the Process:

  Continue the upscaling process by generating the image again. Essentially, you’re using the upscaled image from the previous step and upscaling it further.

• Iterative Enhancement:

  Repeat steps 3 and 4 as many times as necessary to achieve your desired resolution and clarity.

By adopting the Stepping Method, users often find that they can achieve a more refined and detailed image than with a single upscale. This is due to the compounded enhancements made during each iterative step.