Vision systems have been around a long time in laboratory automation, and before that in industrial automation. The Lab Man remembers in the late 80's watching a video made by Gary Kramer (NIST) where he attached a then-small camera to the bottom of a Zymate robot arm to film the robots-eye view of an automated process. The video nearly induced motion sickness among the audience! The purpose wasn't just fun, but to develop visual error-checking mechanisms in the robot interface to a Hewlett-Packard autosampler. Since then, robot-guiding machine vision has been used in a number of colony-picking robot applications, such as the Genetix QPIX, but has never really become a common component for guiding or checking movements in laboratory robotic systems.

With the growth of PC-based computing power and the decrease in the cost and size of CCD or CMOS imaging devices, image-based detection has become common in High Content Screening and molecular pathology. Some of these applications track visual events over the course of time rather than just one static image, capturing up to 30 images per second, which equates to immense amounts of data. Recently The Lab Man came across an application of vision systems and innovative software that is facilitating a completely new way to log and analyze something that has been of interest for a long time - animal behavior. To learn more, we talked to Dr. Lucas Noldus, founder and CEO of Noldus Information Technology in the Netherlands. They're involved in a number of efforts to analyze animal or human behavior using vision systems and novel software, including the evaluation of human expression and body language, and the analysis of animal gait patterns to identify locomotor defects.

The application that caught the "vision" of The Lab Man was the analysis of mouse or rat behavior as the phenotypic expression of the animals genotype, and subsequently evaluating how that behavior varies as the result of disease, surgical treatment, drug treatment or genetic variation. Dr. Noldus points out that pharmacologists and neuroscientists have a long history of devising mechanisms to stimulate and log mouse behavior, usually based on fixed sensors in a special enclosure designed to evaluate activity or locomotion. This has provided a rather single-dimension insight into very complex behavior and it was often hard to separate behavior as a response to the unique and stressful test surroundings vs. behavior due to drug or disease. Noldus supposed that it would be much better to evaluate changes in animal behavior as they occurred in their regular "home" without the overt intrusion of sensors or devices. Home for a laboratory mouse means their home cage, complete with familiar water, food and bedding.

To accomplish this "low intrusion" behavioral analysis, one must be able to observe and log very subtle changes in animal behavior that may occur in the home cage environment, such as types of body postures, body motions, interactions with other animals and interactions with their environment. Vision systems allow the capture of raw data with enough resolution and detail to observe such subtle behaviors. Noldus has designed home animal cages that unobtrusively incorporate a wide field video camera to capture this data. He and his colleagues then needed to develop software that was capable of isolating and logging subtle behaviors. This required two breakthroughs. In the past, image analysis has required relatively static environments, with stationary test objects and clean, simple, high contrast and reproducible backgrounds to ensure reliable isolation of the test subject image. However, home cage environments present a very dynamic image situation, with bedding, substrate and perhaps food scattered around in ever-changing patterns. The animal itself moves around and may appear differently at various times or angles due to spots or fur texture. So Noldus developed dynamic subtraction algorithms to allow their image analysis software to isolate and monitor the test animals image over long periods of time in the midst of a dynamic environment. Secondly, they developed software tools to track multiple points on the contour of the animal to get a more accurate analysis of various behaviors beyond just simple locomotion and position in space.

This software all runs easily on a typical, high-end PC. In fact, Noldus has designed a high-throughput version where one CPU is used to acquire data from four cage modules. Dozens of these can be networked together to automate an entire animal facility.

For those of you who like to visualize, go to this web page to view some interesting video clips of this technology in action. And be aware, as you sit at your desk doing this, that a vision system may be evaluating your behavior pattern!

Until next time,

Domo Arigato, Mr. Roboto