By Staff Writer Benjamin Pfeffer.
An Artificial Intelligence, or AI, software was just created to be able to turn poor sketches into photorealistic drawings of the landscape. Remember those sketches you used to make in Microsoft Paint? You can now turn those into photorealistic drawings or paintings of the landscape, and soon possibly interiors and buildings.
NVIDIA invented this groundbreaking software. The AI software that has the ability to do this is called GauGAN. The GauGAN image creator was named after the French Post-Impressionist painter Eugène Henri Paul Gauguin.
Gauguin lived just over a century ago from 1848-1903. He remains a famous painter for his era and has produced many famous landscape paintings, which is why they decided to name the software after the Post-Impressionist painter… along with the fact that his name almost matched the acronym that NVIDIA was trying to use for the software.
The software uses “generative adversarial networks,” or GAN, to transform even very poor sketches or doodles into photorealistic landscapes.
NVIDIA described the technology behind the software as a “smart paintbrush.” However, it is a deep learning AI trained on a million images using neural networks. The GauGAN image creator is an innovation in artificial intelligence, especially for art.
As Engadget.com stated, apps have been created in the past like Prisma to utilize AI-powered filters and turn photos into paintings that resemble works by Picasso or Van Gogh. Google and Facebook also have this “style transfer” on their platforms.
However, this new creation from NVIDIA takes this style transfer to the next level and creates lifelike pieces from even the most basic outlines. It basically creates something from nothing.
GauGAN is a fairly easy software to use. It has three parts; a paint bucket, a pen, and a pencil. At the bottom of the screen are labels to determine which part of the landscape it is, which includes, sky, tree, cloud, mountain, grass, sea, river, rock, plant, sand, snow, water, and many more.
The user simply selects the label they wish to draw, and the software turns what you draw into an amazing, photorealistic version of your sketch. Lines turn into rivers, circles into clouds, and lumps into cliffs.
According to Engadget, the Vice President of applied deep learning research at NVIDIA describes the software as, “a coloring book picture that describes where a tree is, where the sun is, where the sky is. And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colors, based on what it has learned about real images.”
The software is also able to cast reflections to increase the photo-realism of the landscape. The user can even choose to switch the setting from “grass” to “snow” and change the entire landscape as to if it were the same landscape, but in the winter-even with the leaves fallen off the trees.
There are downsides to the current state of GauGAN. It is currently only a demo because it is showing off its strengths. It would be much more difficult to create furniture and buildings because they are man-made objects and man-made objects allow for much less randomness that can easily be accounted for when creating a nature landscape.
Despite this, NVIDIA still hopes that GauGAN will eventually be good enough to make it too their AI playground. Their AI playground is a new website that opens up their image editing, styling, and photorealistic synthesis software demos to the public. NVIDIA envisions their GauGAN software to be the base tool used by everyone from architects to urban planners to interior designers to video game developers.