Zero-Shot Semantic Style Transfer for Images

Project Abstract

In the field of computer vision, the ability to imbue images with new styles without compromising their semantic integrity remains a challenging and intriguing problem. This process, known as semantic style transfer, aims to apply the stylistic elements of one image onto the content of another while preserving the original content’s meaning and context. The complexity of this task is amplified when it comes to zero-shot style transfer, where the model must generalize to styles never encountered during training. This problem is not just an academic pursuit; it has practical implications in diverse domains such as art creation, design, and augmented reality, where the demand for personalized and context-aware image transformations is growing. However, zero-shot learning in style transfer poses significant hurdles, primarily due to the intricate interplay between content preservation and style application across a potentially infinite style domain.

Given the content image, the semantic segmentation mask of the desired region to be stylized, and the style image, our method proposes to generate a stylized image with arbitrary stylization applied only to the segmented region, in a zero-shot manner.

This project is still under development. More details and results will be updated soon.