Today we are checking out NVIDIA GauGAN2 online demo as well as NVIDIA Canvas, a downloadable AI art creation tool powered by machine learning. GauGAN is named after artist Paul Gauguin and GAN, or generational adversarial network, a popular machine learning algorithm. We looked at the first iteration of GauGAN the beginning of 2021 when we checked out AI Powered Art Tools. The recently released GauGAN 2 adds some interesting new functionality, specifically text based input.
Details of the GauGAN 2 release from the NVIDIA developer blog:
GauGAN2 combines segmentation mapping, inpainting and text-to-image generation in a single model, making it a powerful tool to create photorealistic art with a mix of words and drawings.
The demo is one of the first to combine multiple modalities — text, semantic segmentation, sketch and style — within a single GAN framework. This makes it faster and easier to turn an artist’s vision into a high-quality AI-generated image.
Rather than needing to draw out every element of an imagined scene, users can enter a brief phrase to quickly generate the key features and theme of an image, such as a snow-capped mountain range. This starting point can then be customized with sketches to make a specific mountain taller or add a couple trees in the foreground, or clouds in the sky.
It doesn’t just create realistic images — artists can also use the demo to depict otherworldly landscapes.
Imagine for instance, recreating a landscape from the iconic planet of Tatooine in the Star Wars franchise, which has two suns. All that’s needed is the text “desert hills sun” to create a starting point, after which users can quickly sketch in a second sun.
It’s an iterative process, where every word the user types into the text box adds more to the AI-created image.
Additionally we check out NVIDIA Canvas, a downloadable application for NVIDIA RTX card owners, that using the GauGAN algorithms to enable AI assisted image generation. The new GauGAN2 text driven functionality is not in Canvas yet unfortunately.
Key Links
You can learn more about GauGAN 2 and see it and NVIDIA Canvas in action in the video below.