Local texture-based color transfer and colorization

An extension of our Expressive 2016 paper was published in a special edition of Computers & Graphics this month. This extended paper can be found here.

The main contribution of this extension is the ability to use simple strokes to define image regions and apply color transfer or colorization between these regions only. The image regions are automatically computed from the user provided strokes using our edge-aware texture descriptor.

Here is an example of using these strokes to quickly refine the result of the automatic color transfer approach. First, we compute the automatic color transfer result based on textural properties using the house input image, and the sunset image as a reference.

We are happy with the purplish color of the house and hedge, however the sky remained blue because of the blue sky in the reference. To get a better sunset color in the sky, we use another reference image and two strokes to match the skies of the two images.

As we can see, the masks automatically computed from the stroke (bottom right) accurately separate the sky in the two images. In the final result, only the sky is changed to the purple color of the new reference’s sky, producing a better sunset feeling.