Facebook unveils AI model that copies text style from images using just one word
The past few years have taught us that our faces, voices and lips can be copied and reproduced using artificial intelligence. Now, an AI model created by Facebook researchers can mimic, edit, and replace handwritten and stage text using a single word in an image.
Facebook unveiled TextStyleBrush, an AI research project, Friday. The company points out that while most AI systems can replicate and replace text for well-defined and specialized tasks, TextStyleBrush is different because it can reproduce text both in handwriting and in real-world scenes. . Doing this is much more difficult for an AI model due to the different text options and nuances involved.
“It means understanding unlimited text styles not only for different typography and calligraphy, but also for different transformations, such as rotations, curved text and the distortions that occur between paper and pen when handwriting; background bulk; and image noise, ”Facebook explained in a press ad.
TextStyleBrush is the first self-supervised AI model that can replicate and replace text in handwritten and real-world scenes using a single sample word from an image, according to Facebook. It has displayed great accuracy in automated testing and user studies for any type of text, the company said, adding that it intends to submit the work to a peer-reviewed journal.
The AI model works in a similar fashion to the brush-style tools used in word processors. In this case, the method is applied to the aesthetics of the text in the images. However, Facebook researchers decided to forgo traditional methods, which include setting specific parameters such as overseeing target typeface or style, with TextStyleBrush.
“[W]We take a more holistic approach to training and decouple the content of a text image from all aspects of its appearance from the entire box of words, ”Facebook said. “The overall appearance representation can then be applied as a one-time transfer without recycling onto the new source style swatches. “
TextStyleBrush is not without limits, however. Facebook said this included working with text written in metallic objects or text with characters in different colors.
While at first glance this AI model seems like a cool and efficient way to copy and replace text, it shouldn’t be seen as just an innocent tool, either. One of the first things I thought about, for example, was how TextStyleBrush could be used to deceive others or even commit crimes.
Facebook recognizes that deepfake text attacks are a problem, and said he hoped that by openly posting his research to TextStyleBrush, he could encourage further research to anticipate these attacks. The company said that if researchers and AI practitioners can develop this technology before the bad actors, they can “learn to better detect this new style of deepfakes and build robust systems to combat them.”