Incremental Learning for Visual Tracking


Most existing tracking algorithms construct a representation of a target object prior to the tracking task starts, and utilize invariant features to handle appearance variation of the target caused by lighting, pose, and view angle change. In this paper, we present an efficient and effective online algorithm that incrementally learns and adapts a low dimensional eigenspace representation to reflect appearance changes of the target, thereby facilitating the tracking task. Furthermore, our incremental method correctly updates the sample mean and the eigenbasis, whereas existing incremental subspace update methods ignore the fact the sample mean varies over time. The tracking problem is formulated as a state inference problem within a Markov Chain Monte Carlo framework and a particle filter is incorporated for propagating sample distributions over time. Numerous experiments demonstrate the effectiveness of the proposed tracking algorithm in indoor and outdoor environments where the target objects undergo large pose and lighting changes.


For each panel, the number on the upper left corner denotes the frame number, and the tracking results are shown in the first row, where the green boxes represent particles with large confidence and the red box is the final estimate.
The images in the second row shows the current sample mean, tracked image region, reconstructed image with the mean and eigenbases, and the reconstruction error respectively. The third and forth rows show 10 largest eigenbases.
Indoor Sequence
A person moves from dark toward bright area with large lighting and pose changes.

result: [ avi(16MB) ]

Doll Sequence
An animal doll moving with large pose, lighting variation in a cluttered background. The first result shows two failure-and-recovery in severe pose changes before it loses track after 900th frame. The second result with forgetting factor, it gives more robust tracking result.

result: without forgetting factor [ avi(31MB) ]   with forgetting factor [ avi(32MB) ]

Outdoor Sequence
A person moves underneath a trellis with large illumination change and cast shadows while changing his pose. The tracker works well over the severe shadows and illumination changes. Without forgetting factor, it fails when the person changes his pose suddenly while the lighting is also changing. The second result with forgetting factor does not fail, but the drift in scale and rotation is noticeable.

result: without forgetting factor [ avi(12MB) ]   with forgetting factor [ avi(17MB) ]

Dudek Sequence
This sequence is originally used in Jepson et al.'s tracking paper. We subsampled the original video of 30fps into 15fps to give faster motions, and our tracker could successfully track it. This sequence includes pose changes, facial expression and appearance (glasses) changes.

result: without forgetting factor [ avi(34MB) ]