Email this Article Email   

CHIPS Articles: Research and Development or Algorithms for Automated Video Surveillance

Research and Development or Algorithms for Automated Video Surveillance
Increasing automated system accuracy by improving algorithms used for digital motion estimation.
By SSC Charleston - April-June 2008
As the use of digital video for surveillance increases, there remains a need to automate and improve these systems, not only to decrease the amount of manpower required to operate surveillance systems, but also to increase their capability. This article details the evolution and improvement of extracting "moving" objects from an original digital video input, such as an AVI or MPEG file.

SPAWAR Systems Center (SSC) Charleston is researching, developing and simulating various algorithms used to automate motion detection in digital video. As the capabilities of digital video surveillance increase, the systems will not only rely on improving hardware, such as processor speed and memory size, but also the use of intelligent computing.

In the past five years, software systems have been developed that allow automated video surveillance that can be performed independently without human analysis. Video motion detection can recognize motion using methods such as block matching, edge detection and optical intensity flow analysis.

Block matching is a standard technique for encoding motion in video compression algorithms. Block matching is a means of correlating a block from the first image with a block of the second. The position at which the first block correlates in the second image is then later used to infer data.

Edge detection uses algorithms to mark the points in a digital image at which the luminous intensity changes sharply. The principle here is that sharp changes in image properties usually reflect important events and changes in the properties of the actions depicted.

The optical intensity flow analysis method uses the optical flow in a selective attention model for object tracking. It focuses on the interested regions of an image. This method uses results for identifying an interesting object and tracks the target using the optical flow information when the identified object moves.

But these methods could be considered obsolete compared to the systems in use today.

Methods of detecting motion are becoming increasingly advanced and are better able to discern content and activities. Companies are using advanced video content analysis to detect moving objects, while suppressing unwanted alarms from misleading sources in the image. This means that the algorithm intelligently adapts to changing lighting and environmental conditions such as rain, snow and leaves blowing in the wind.

Recent advances include sophisticated boundary control, integrated camera panning to automate tracking of the object in motion, and increasingly important, color identification along with gait and motion pattern classification.

The challenge to provide a reliable automated video surveillance system remains; however, because the system is dependent on the digital video and the effects of the environment in that video.

In 2006, SSC Charleston conducted an evaluation of commercial automated video surveillance systems by monitoring a shipping harbor. The environment captured on the video caused the systems to output false alarms. Some of the nuisances found in the testing were caused by the wind blowing leaves on trees, unintentional movement by the camera due to the wind and changes due to light reflection in the moving water.

In 2007, SSC Charleston conducted additional evaluations of automated video surveillance systems paired with night vision technology, such as high performance monochrome, image intensifiers and thermal imagers. Results of the evaluations showed that algorithms needed to be improved.

Using MATLABĀ® (matrix laboratory) and SimulinkĀ® (simulation and model-based design) to model and simulate the algorithms, SSC Charleston is able to test the proficiency of the algorithms before porting them to embedded digital signal processors (DSP).

DSP is a special type of coprocessor designed for performing mathematics. Most DSPs are programmable, which means that they can be used for manipulating different types of information, including sound, images and video.

MATLAB is a high-level technical computing language and interactive environment for algorithm development, data visualization, data analysis and numeric computation.

Using MATLAB, you can solve technical computing problems faster than with traditional programming languages, such as C, C++ and Fortran. You can integrate your MATLAB code with other languages and applications and distribute MATLAB algorithms and applications.

MATLAB allows mathematical functions for linear algebra, statistics, Fourier analysis, filtering, optimization and numerical integration.

Simulink is an environment for multidomain simulation and model-based design for dynamic and embedded systems. It provides an interactive graphical environment and a customizable set of block libraries that let you design, simulate, implement and test a variety of time-varying systems, including communications, controls, signal processing, video processing and image processing.

One specific algorithm that SSC Charleston inspected was the Sum of Absolute Differences (SAD), as detailed by E.A. Hakkennes of the Delft University of Technology in the Netherlands. SAD is one of the more effective algorithms in current automated video surveillance systems.

SAD is a summation of the absolute differences between pixels in an image. In this algorithm, each pixel is compared with every pixel from the next image (one frame delay). The sum of these calculations corresponds to a value which can be used to form an output matrix (video) with only the changed (moving) pixels. The formula is shown at right.

A model of SAD was created in Simulink. A sample video AVI/MPEG file was used as input and the numbering system was changed from a fixed point 8-bit integer to a 16-bit integer (int16) for compatibility with the SAD algorithm.

Further development is needed to supplement these algorithms using noise filters and supporting functions such as marking, tracking and alerting.

One way to filter out water-caused nuisances is to develop a water classification engine that characterizes the behavior and effects of the natural environment on water.

SSC Charleston is particularly focused on reducing wake detection, so that a more accurate bounding box containing the real object becomes the output, which gives the user a better object size estimation, classification, and other characteristics used to identify an object of interest.

Due to the complex characteristics of the water environment, an algorithm to handle water reflections, shadows, waves and wakes should also be used with the motion detection algorithm.

SSC Charleston is currently investigating new algorithms that use subbands from the frequency domain in addition to the current methods. Subband coding breaks a signal into a number of different frequency bands and encodes each one independently. This decomposition is often the first step in data compression for audio and video signals.

Future video surveillance systems will take advantage of intelligent computing concepts such as the use of state theory and artificial neural networks.

State theory is a theory of absolute time in which the number of states from the first moment in time up to the current moment in time can be counted.

In contrast with Einstein's theory of relativity, in which time is modeled using the real number system, state theory defines a state as a moment in time, a point of time, an instant, and as such has no duration. State theory and binary logic overlap somewhat due to the definition of a statement. In binary logic, a statement is a sentence which is either true or false, but cannot be true and false simultaneously. That is, no statement can be true and false in the same state.

An artificial neural network (ANN), also called a simulated neural network (SNN), or commonly just neural network (NN), is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionistic approach to computation.

In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network. In more practical terms, neural networks are non-linear statistical data modeling or decision making tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data.

Because of the higher level of analysis and adaptation gained with these two methods, intelligent systems can be developed that will not only be able to characterize objects of interest from nuance; but will also be able to learn and recognize similar objects from the past.

Analysis and modeling for this work was conducted by Joey F. Pomperada of SSC Charleston. Field work and operational assessment was conducted by Justin Firestone of SSC Charleston.

Development was completed as part of the 2006 SSC Charleston Innovation Program, an ongoing command initiative to assist the warfighter by encouraging professional growth in cutting edge areas of technology and business.

For information about SSC Charleston, go to the SPAWAR homepage, www.spawar.navy.mil.

TAGS: ITAM
Before and after shot of detecting motion using the SAD algorithm for automated motion detection in digital video. The photo above is an image from a video. The image below is the same image after application of the SAD algorithm. SAD is a summation of the absolute differences between pixels in an image. In this algorithm, each pixel is compared with every pixel from the next image (one frame delay). The sum of these calculations corresponds to a value which can be used to form an output matrix (video) with only the changed (moving) pixels.
Before and after shot of detecting motion using the SAD algorithm for automated motion detection in digital video. The photo above is an image from a video. The image below is the same image after application of the SAD algorithm. SAD is a summation of the absolute differences between pixels in an image. In this algorithm, each pixel is compared with every pixel from the next image (one frame delay). The sum of these calculations corresponds to a value which can be used to form an output matrix (video) with only the changed (moving) pixels.

This image is the same image as above after application of the SAD algorithm. SAD is a summation of the absolute differences between pixels in an image. In this algorithm, each pixel is compared with every pixel from the next image (one frame delay). The sum of these calculations corresponds to a value which can be used to form an output matrix (video) with only the changed (moving) pixels

In this algorithm, each pixel is compared with every pixel from the next image (one frame delay).
In this algorithm, each pixel is compared with every pixel from the next image (one frame delay).

CHIPS is an official U.S. Navy website sponsored by the Department of the Navy (DON) Chief Information Officer, the Department of Defense Enterprise Software Initiative (ESI) and the DON's ESI Software Product Manager Team at Space and Naval Warfare Systems Center Pacific.

Online ISSN 2154-1779; Print ISSN 1047-9988
Hyperlink Disclaimer