Self-Supervised Multi-Scale Pyramid Fusion Networks for Realistic Bokeh Effect Rendering

To balance the aesthetic requirements on photo quality and expensive high-end SLR cameras, synthetic bokeh effect rendering has emerged as an attractive machine learning topic. However, most of bokeh rendering models either heavily relied on prior knowledge or were topic-irrelevant data-driven networks without task-specific knowledge, which restricted models' training efficiency and testing accuracy. Therefore, in this paper, following the principle of bokeh generation, a novel self-supervised multi-scale pyramid fusion network for bokeh rendering has been proposed. Structure consistencies are employed to emphasize the importance of bokeh components. Task-specific knowledge which mimics the "circle of confusion" phenomenon through disk blur convolutions is utilized as self-supervised information for network training. The proposed network has been evaluated with several state-of-the-art methods on a public large-scale bokeh dataset- the "EBB!" Dataset. The experiments demonstrates that it has much better processing efficiency and realistic bokeh effect with much less parameters size and running time