Inpainting – the history and possibilities of one of the most impressive AI sectors
Many paintings from the past end up in museums where they get damaged. This is when a trained art restorer comes in and carefully studies the work, identifies the techniques and tools used to create it, and finds the best way to repair the masterpiece.
⠀
This is a traditional method, but artificial intelligence solutions have made it possible to transfer it to the digital world. Machine learning outsourcing, which is one of the largest branches of AI systems, uses the inpainting method. One could even say that it is a digital art restorer. But what exactly is inpainting and how did it begin?
⠀
What is inpainting and how important is artificial intelligence to it?
⠀
In the world of digital image processing, there are many tools available for both editing and reconstructing images. All by dint of artificial intelligence and deep learning, which proves that nothing is yet a foregone conclusion when it comes to repairing or even completely recovering parts of photographs. The implementation of AI in countless aspects of our lives allows it to expand the scope of possibilities it offers on an extraordinary scale. So let’s shed light on where the idea for inpainting stems from.
⠀
Inpainting – digital image preservation
⠀
Inpainting is the process of painting and photograph conservation in which damaged, destroyed or missing pieces are filled in to create a complete image. The process can be applied to both physical and digital works.
⠀
Artificial intelligence solutions have no problem with any type of art, and are perfect for restoring oil paintings, acrylic paintings, photographic prints, sculptures, digital images or even videos. AI design aimed at restoring paintings can be found, for example, while using the most popular graphic editing tools delivered by Adobe.
⠀
⠀
Artificial intelligence is capable of repairing images using two methods: non-blind inpainting and blind inpainting. The first method involves filling in parts of an image using an AI implementation or, more specifically, a certain algorithm. It can also prove effective in removing text, scribbles and even large objects from a selected image or photograph.
⠀
Blind inpainting, on the other hand, is a method that is used when a piece of the image is completely missing. It also works on the basis of an algorithm that an AI programmer has created, but it slightly differs from non-blind inpainting. Here, the algorithm must be able to automatically identify and fill in the gaps in the image using data analysis. As you might expect, this is a demanding and difficult method.
⠀
History of inpainting
⠀
It may come as a surprise that inpainting has been around for quite some time. It was not until the International Conference in 1930 that a modern approach to painting was created. Helmut Ruhemann, a leading expert in the modernisation, restoration and conservation of works of art, attempted to imitate exactly the methods of the original painter and understand his artistic intentions.
After many years, Cesare Brandi developed a theory called del restauro, which combined both aesthetics and psychology. It was mainly used by Italian conservators and restorers, and the terminology itself became widely known in the 1990s.
⠀
AI systems – on what basis does inpainting work?
⠀
In the 1990s, when more premium was placed on artificial intelligence solutions than ever before, some new applications were introduced to inpainting. This made use of artificial intelligence, allowing for an extensive use of digital techniques based on deep learning and machine learning technologies.
⠀
Inpainting in the 1990s included both computerised painting and the use of tools to manually stimulate the restoration process. The process was known as image or video interpolation. Digital painting involved the use of algorithm-based computer software to replace lost or damaged parts of the image data. It all came about through the implementations of AI, as well as its algorithms for further possible applications.
⠀
⠀
Methods of using inpainting
⠀
AI systems, due to their growth, have enabled several different ways of its use in inpainting. Artificial intelligence solutions have been introduced into many programs that allow for the reconstruction of missing or damaged areas of digital photos and videos.
⠀
Apart from filling in missing elements, it is also possible to use this tool to remove text, scratches, or objects from photos, and even use inpainting to modify films. It is also great for fixing film damage, removing red-eye or even erasing individual objects to achieve the desired effect.
⠀
But let’s get to the point: what digital inpainting methods can be used?
⠀
Structured painting
⠀
It is a method that is applicable to smooth images that have defined contours. It works on the principle of being able to recover the geometry of a photograph by using similar areas. Its work can be considered to be similar to that of data scientists.
Bertalimio proposed a method of structural painting that mimics the way trained conservators deal with the restoration of paintings in physical form. The method’s purpose is to move similar parts of the painting gradually into the gap. This way, it can only be filled by the existing parts of the painting.
⠀
Texture technique
⠀
In the case of images that have varying textures, the textural method seems perfect. It has a repeatable pattern, which means that the missing part of the image cannot be restored by continuing the level lines inside the gap. In order to repair a given spot, AI systems collecting frequency and spatial information are employed, allowing the gaps to be filled with the desired texture.
Interestingly, although this is the simplest method, it is also sometimes used when selecting a texture for painting. In the case of a texture covering a larger area or a bigger frame, segmentation of the parts to be painted should be used by selecting appropriate textures from the whole image.
⠀
Combined method
⠀
The combination of the two above procedures intends to fill in the texture and complete the structure of the images in the gaps. Example methods created by AI developers help automatise the process of cloning parts of an image.
The filling is performed by searching for similar locations in the source area and copying the pixels with the highest similarity to the missing part. Doing the filling with a patch, rather than pixels, allows the algorithm to reduce the blurring caused by the previously used techniques.
⠀
Summary
⠀
Artificial intelligence solutions make what was once impossible into reality. Thanks to skilled AI developers, filling in gaps or removing disturbing elements in a photo has become so simple that it can be done by almost every user. Due to the algorithms used to create tools, such as a patch or move with content available in many applications, repairing an image or photograph is no longer that much of a challenge.
⠀
AI systems, because of their level of advancement, are able to perform in digital form what professional image restorers can do in handwritten form. Have a look at our offer for digitising businesses and increasing their efficiency with marketing data analytics.