Artistic neural style transfer using CycleGAN and FABEMD by adaptive information selection

Abstract

Neural Style Transfer (NST) comprises a class of computer vision methods that manipulate digital images to reformulate the visual content of one input image adopting visual features of an another image set. Artistic NST is the particular case of NST where the visual features are extracted from painting images. The combination of Cycle-Consistent Adversarial Networks (CycleGANs) with Fast and Adaptive Bidimensional Empirical Mode Decomposition (FABEMD) is proposed to adopt the specific artist’s style on images effectively, where the cycle-consistency loss is modified to incorporate texture information by estimating the corresponding Bidimensional Intrinsic Mode Functions (BIMFs). An adaptive approach for identifying the optimal BIMF number that must be considered in order to manipulate the required amount of the involved texture, is proposed. For this purpose, the computation of a metric is considered for each BIMF to characterise the texture of each image or major intensity alterations at local scale. Experimental results reveal that adaptive comixture of texture features comprises an efficient approach in such artistic applications. Qualitative and quantitative results demonstrate that the proposed framework outperforms state-of-the-art (SoA) methods.