The current generation of visual sensors consist of a homogenous array of receptors. However, it is notable that biological systems have a different architecture with a high resolution foveal region and a lower resolution surround. Bottom-up or top-down attentive processes orient the fovea to areas of interest.

One possibility would be to develop a camera system which has spatially inhomogenous sampling. However, a similar effect can be acheived by having two cameras (top right). One has a wide-field or panoramic field of viewand is fixed in position. The second camera is has a narrower field of view and is attached to pan and tilt motors. Since the same number of pixels are spread across different fields of view, the mobile camera provides much more detail.

We fuse the data from the two views by finding the best homography relating the two images, and warping the foveal ouput on top of the paranoramic view (left). The position of the foveal sensor can be manipulated by clicking on a given position in the lower field image.

We are developing algorithms to make the attentive process automatic. These combine bottom up (background subtraction, motion differencing, face detection) measurements and top-down constraints such as a tendency to track objects, and to get bored of looking at the same object all the time. The result is that the sensor automatically attends to the people in the scene. The system records snapshots of the people in the room.

This system has potential applications in security and e-learning. Further details avalable from the main Elderlab website.

Sensor consists of wide field camera(bottom) plus high-res foveal sensor attached to motors (top)



Output of low-resolution and high resolution sensors are dynamically combined in real time.

DOWNLOADABLE MEDIA

• Attentive Sensor Tracking Video 1 (4.0 Mb, MPEG, Canada)
• Attentive Sensor Tracking Video 2 (1.5 Mb, MPEG, Canada)
• Camera Tracking Video (2.2Mb, MPEG, Canada)
• Snapshots Video (3.6Mb, AVI, Canada)

PUBLISHED WORK

• S.J.D. Prince, J. H. Elder, Y. Hou and Y. Oleviskiy, “Statistical cue integration for foveated wide-field surveillance,” In Proc. IEEE Computer Vision and Pattern Recognition, 2005.
• J. H. Elder, S.J.D. Prince, Y. Hou, M. Sizintsev and Y. Oleviskiy, “Pre-Attentive and Attentive Detection of Humans in Wide-Field Scenes”, International Journal of Computer Vision, Vol. 72, pp. 47-66, 2007.
• S.J.D. Prince, M. Sizinstev, B. Hou and J. Elder. “Pre-Attentive Face Detection for Foveated Wide-Field Surveillance,” IEEE Workshop on Applications in Computer Vision, pp. 439-446, 2005.
• J. Elder, Y. Hou, S.J.D. Prince, M. Sizinstev, "Pre-Attentive Face Detection," 14th Annual Canadian Conference on Intelligent Systems, 2004. (Poster)