This paper presents a novel image smoothing approach using a space-filling curve as the reduced domain to perform separation of edges and details. This structure-aware smoothing effect is achieved by modulating local extrema after empirical mode decomposition; it is highly effective and efficient since it is implemented on a one-dimensional curve instead of a two-dimensional image grid. To overcome edge staircase-like artifacts caused by a neighborhood deficiency in domain reduction, we next use a joint contrast-based filter to consolidate edge structures in image smoothing. The adoption of dimensional reduction makes our smoothing approach distinct for two reasons. First, overall structure-awareness is improved as more extrema are exploited to locate the salient edges and details. Second, envelope computation for local extrema is made much fast by using explicit interpolants on the curve. Moreover, our approach is simple and very easy to implement in practice. Experimental results demonstrate the merit of our approach, which outperforms previous state-of-the-art methods, for a variety of image processing tasks.

In this paper we introduce SceneGraph, a novel non-linear representation for scene structure of a video. Unlike classical linear sequential organization, SceneGraph concentrates the video content across the time line by structuring scenes and materializes with two-dimensional graph, which enables non-linear exploration on the scenes and their transitions. To construct SceneGraph, we adopt a sub-shot induced method to evaluate the spatio-temporal similarity between shot segments of video. Then, scene structure is derived by grouping similar shots and identifying the valid transitions between scenes. The final stage is to represent the scene structure using a graph with respect to scene transition topology. Our SceneGraph can provide a condensed representation in the scene level and facilitate a non-linear manner to browse videos. Experimental results are presented to demonstrate the effectiveness and efficiency by using SceneGraph to explore and access the video content.

This paper presents a novel video stabilization approach by leveraging the multiple planes structure of video scene to stabilize inter-frame motion. As opposed to previous stabilization procedure operating in a single plane, our approach primarily deals with multiplane videos and builds their multiple planes structure for performing stabilization in respective planes. Hence, a robust plane detection scheme is devised to detect multiple planes by classifying feature trajactories according to reprojection errors generated by plane induced homographies. Then, an improved planar stabilization technique is applied by conforming to the compensated homography in each plane. Finally, multiple stabilized planes are coherently fused by content-preserving image warps to obtain the output stabilized frames. Our approach does not need any stereo reconstruction, yet is able to produce commendable results due to awareness of multipe planes structure in the stabilization. Experimental results demonstrate the effectivenss and efficiency of our approach to robust stablization on multiplane videos.

We introduce the EXtract-and-COmplete Layering method (EXCOL)—a novel cartoon animation processing technique to convert a traditional animated cartoon video into multiple semantically meaningful layers. Our technique is inspired by vision-based layering techniques but focuses on shape cues in both the extraction and completion steps to reflect the unique characteristics of cartoon animation. For layer extraction, we define a novel similarity measure incorporating both shape and color of automatically segmented regions within individual frames and propagate a small set of user-specified layer labels among similar regions across frames. By clustering regions with the same labels, each frame is appropriately partitioned into different layers, with each layer containing semantically meaningful content. Then, a warping-based approach is used to fill missing parts caused by occlusion within the extracted layers to achieve a complete representation. EXCOL provides a flexible way to effectively reuse traditional cartoon animations with only a small amount of user interaction. It is demonstrated that our EXCOL method is effective and robust, and the layered representation benefits a variety of applications in cartoon animation processing.

Collage can provide a summary form on the collection of photos in an album. In this paper, we introduce a novel approach to constructing photo collage in the hierarchical narrative manner. As opposed to previous methods focusing on spatial coherence in the collage layout, our narrative collage arranges the photos according to the basic narrative elements from literary writings, i.e., character, setting and plot. Face, time and place attributes are exploited to embody those narrative elements in the collage. Then, photos are organized into the hierarchical structure for the multi-level details in the events recorded by the album. Such hierarchical narrative collage can present a visual overview in the chronological order on what happened in the album. Experimental results show that our approach offers a better summarization to browse on the photo album content than previous ones.