Getting Personal: Facial Recognition Spots the Suspects

Imagine a video surveillance system that automatically watches for known suspects.
Publish date:
Updated on

Imagine a video surveillance system that automatically watches for known suspects. That, in a nutshell, is what is created by feeding CCTV surveillance feeds to facial recognition software.

by James Careless

Together, this system compares faces in its database with images caught on real-time video; comparing distances between the eyes and other facial “landmarks” for similarities. When enough similarities occur, the system automatically alerts security personnel, allowing the humans to react to these threats quickly.

Image placeholder title

Multiple facial recognition using VeriLook SDK, from Neurotechnology and available through Fulcrum Biometrics. Granted, integrated face recognition/video surveillance systems are not perfect—false positives do happen. Still, “Accuracy has improved nearly two full orders of magnitude since the large scale studies [that were] published in 2002,” says Ken Nosker, president of Fulcrum Biometrics. “In the latest independent study published by NIST, researchers have shown that seven tested algorithms performed as good as or better at matching faces than humans were able to do.

“Major improvements to facial recognition performance have come from three key areas,” he adds. These are “higher resolution imaging devices, the introduction of 3D face recognition, and new image preprocessing techniques for handling variations in lighting or illumination before passing the processed images to the face recognition algorithms.”

For law enforcement, military and other government agencies who rely on video surveillance, facial recognition is indeed a powerful tool. But it is a tool with limits that must be acknowledged and dealt with, before this technology can perform as promised.


The human face is like a fingerprint: Even between twins, there are differences in facial features that are measurable and storable. These include the distance between the eyes, mouth, nose and other elements. They also encompass the size and shape of features by themselves and in relation to other facial landmarks.

These measurements are captured by facial recognition software, then crunched and compacted by algorithms to reduce the complexity of the facial comparison process. When an image from a video feed is subsequently captured, it too is analyzed algorithmically and then compared against the facial recognition database; just as a suspect’s fingerprint is analyzed against a fingerprint database.

Like a fingerprint system, a facial recognition system is only as good as the number and quality of facial images stored in its database. This makes it useful for identifying people already in the system, but not those for whom no records exist.


Facial recognition/video surveillance systems are best known for spotting suspects in real time. But that’s not all they can do, says Stephen G. Russell, chairman and founder of 3VR, a San Francisco-based leader in intelligent surveillance. “Facial recognition systems can also be used in a historical sense, to search through recorded surveillance video to document a suspect’s activities over time.”

When combined with an automated alerting system, facial recognition/video surveillance systems can do the “dog work” for human operators, methodically sorting through the frames looking for algorithmic matches. They can also keep a lookout for hundreds and even thousands of suspects simultaneously, and mine the digital archives for these people’s past whereabouts and actions.

“The best part is that modern facial recognition software can work with NTSC-quality images,” says Henry Schneiderman, president of Pittsburgh Pattern Recognition. “In fact, our software can work with even smaller images, with just 25 pixels distance between the eyes. And it will operate on any reasonably powerful Intel-based computer.”

Image placeholder title

3VR System “Our non-realtime Examiner software can also compensate for poor lighting conditions, and model 2D images into three dimensions,” says Kevin Raderschadt, senior sales manager with Cognitec Systems Corp., based in Dresden, Germany. “Examiner gives police Face Examiners the tools to take unusable crime scene video facial images and correct/enhance the face images and match them against their mugshot repositories for the individual’s identity and arrest.”


As mentioned earlier, even the best facial recognition/video surveillance system has some limits.

Some of these have to do with the shooting environment. If the lighting is poor, or there are windows allowing in blinding sunlight, the quality of the image will be compromised. In the same vein, if the camera is covering a large floor space with lots of obstacles and many faces passing by, the software can only do so much with the resulting video.

Other limits are linked to the CCTV surveillance equipment. If the cameras are too low-resolution, or just plain worn out from years of use, the resulting images won’t give the facial recognition software enough to work with. If the cameras are also placed too high above the floor, so that the resulting camera angle looks down steeply at the human subjects, the software will have trouble comparing these images to face-forward mug shots. Finally, if the security organization’s video recordings are still being made to tape, then any historical searching will be restricted either to real-time playback, or have to wait until the video can be digitized into a nonlinear file format.

Processing power can be an issue. If the specific software is asked to do too much on a low-powered computer, delays in processing will occur. In situations where real-time comparisons are a must, the system will simply be unable to deliver in a reliable, timely manner. And don’t forget the quality of the facial image database: If the reference images aren’t good enough in terms of clarity and resolution, then the whole system will be hobbled.

“Given these limits, it appears that facial recognition has a long way to go before it reaches the accuracy and reliability rates of fingerprint or iris-based recognition,” says Nosker, of Fulcrum Biometrics, which has offices in India and San Antonio. “The consensus of industry—and large users of biometric systems such as the Department of Defense—is thus that facial recognition is not sufficient as a standalone determination of identity in large scale databases/populations.” This is why his company sells the Fulcrum Biometrics Framework (FbF) identity solution, “which seamlessly incorporates face, iris, fingerprint and palm print identification.”


All of the limits described above can be dealt with, allowing security organizations to get the most out of their facial recognition/video surveillance systems.

“Lighting is key to capturing high resolution, usable video images,” says Schneiderman of Pittsburgh Pattern Recognition. “If you’ve got windows or other sources of glare in the shot, either mask them out with curtains or move the camera to another location.”

Cameras should also be restricted to medium or even close shots, to give the software more to work with when scanning an individual face. One way to do this successfully is to locate the security position “in a choke point,” says Raderschadt, of Cognitec. “By putting the position in a narrow, restricted area, you reduce the number of people that the camera has to cover; you can even bring it down to just one with proper camera placement.”

Image placeholder title

Cognitec Systems Corp.’s Examiner software corrects and enhances images to help find matches in a database. A second improvement is moving the camera down from their overhead perches, so that they are shooting towards the person being surveilled at eye level. “It makes sense to shoot surveillance video as close to the angle and position of your reference facial images,” Schneiderman says. “This provides a similar sample for the software to compare against the database, enhancing its ability to make accurate matches.”

“Nevertheless, a sample image may not always be from the same angle as the reference image,” he adds. “Our company has made great progress in being to recognize non-frontal face images and will be releasing software that accomplishes this shortly. This software will greatly expand the usability of face recognition to situations where it had not possible in the past.”

So, be sure that your surveillance system is recording to digital nonlinear files, and that these files are compatible with the facial recognition program you are thinking of buying. Use computers that have more than enough processing power to run the software, to allow ‘headroom’ for extra demands and eventual software upgrades. And to play it safe, consider Nosker’s advice about using facial recognition in tandem with other forms of surveillance and identification. Multiple systems provide higher certainty, plus a backup path should your facial recognition computer fail.

The bottom line: Combining facial recognition software with video surveillance can deliver a potent boost to any security system. In today’s justifiably paranoid times, it is an investment worth looking into.