Design and analysis of heterogeneous sensors based object tracking systems.

In a surveillance system for monitoring objects, there is an increasing need for developing robust algorithm as well as dealing with intelligent interaction among different types of sensors and information. This dissertation describes a design and an analysis of object tracking methodology in heterogeneous sensor network. In the first part, we consider the object tracking problem in three dimensional (3-D) space when the azimuth and the elevation of the object are available from the passive acoustic sensor. The particle filtering technique can be directly applied to estimate the 3-D location of the object, but we propose to decompose the 3-D particle filter into the three planes' particle filters which are individually designed for the 2-D bearings-only tracking problems. The 2-D bearing information is derived from the azimuth and the elevation of the object to be used for the 2-D particle filter. Two estimates of three planes' particle filters are selected based on the characterization of the acoustic sensor operation in noisy environment. The proposed approach is extended to multiple acoustic sensors and its robustness is analyzed. The Cramer-Rao Lower Bound of the proposed 2-D particle filter-based algorithm is derived and compared against the algorithm based on the direct 3-D particle filter. In the second part, the object tracking by a single acoustic sensor based on the particle filtering is extended for the multiple objects, and the corresponding inherent limitation is introduced. In order to overcome the limitation of the acoustic sensor for the simultaneous multiple object tracking, the support from the visual sensor with the objects' localization is considered. The cooperation from the visual sensor, however, should be minimized, as the visual sensor's object localization requires much higher computational resources than the acoustic sensor based estimation, and the visual sensor is usually not dedicated to the object tracking and deployed for other applications. The acoustic sensor mainly tracks multiple objects and the visual sensor supports the tracking task only when the acoustic sensor has a difficulty. Several techniques of the particle filtering are used for the multiple object tracking by the acoustic sensor and the limitations of the acoustic sensor are discussed to identify the need of the visual sensor cooperation. Performance of the triggering-based cooperation of the two visual sensors is evaluated and it is compared with a periodic cooperation in a real environment. In the third part, we address enhancement of object detection with multiple visual sensors. The detection enhancement we introduce is to recover missed object detection given partially detected objects among multiple visual sensors. Once an object is detected by one or more visual sensors, the detected local object positions are transformed into a global object position. Based on a local and global information collaboration, any missed local object position is recovered by the global to local transformation. However, the collaboration may degrade the detection performance by incorrectly recovering the local object position, which is propagated from false object detection. Furthermore, local object positions corresponding to an identical object are transformed into inequivalent global object positions due to detection uncertainty such as a shadow. We minimize the performance degradation by preventing from the propagation of the false object detection. In addition, we present an evaluation method for a final global object position. Finally, the proposed method is analyzed and evaluated with case studies. In the last part, we summarize and...