| Network
of Excellence |
CARiMan |
The
Internet Portal for Computer Aided Risk Management |
Dr. Anastasios Doulamis
National Technical University of Athens (NTUA),
Dept. of Electrical & Computer Engineering
Tel: +30-10-7722546
Fax: +30-10-7722534
adoulams@cs.tua.gr
ndoulam@cs.ntua.gr
CARiMan
in Traffic Accidents
Safety
from traffic accidents is probably one of the most important issues for
preventing the risk management. Cars today come with all sorts of safety
features. Side curtain airbags, crumple zones, beam reinforced doors and more
make each new car safer for you and your family. But what is the future of
safety? What can we expect to be paying for in say, 20 years? The same are for
trains, and buses. New features and tools are required to prevent them from an
accident.
In
many ways the future of vehicle safety is already here. However, advanced safety
features are very expensive and in many cases are in their prototype form.
Ultimately, new safety features will take over actions that the driver/pilot
currently performs routinely. There may even come a day when our cars will have
backwards-facing front seats, so we can talk with the people in the back seat
and let the computer do the driving
Computer
vision
Computer
vision tools can significantly help the driver/pilot and increase the safety of
vehicles. Towards this direction, infra-red and/or standard video cameras can be
used to detect problems and help the driver/pilot. Infra red cameras can be
mounted on the front of the vehicle to detect heat sources, and project that
image onto the windshield in a kind of transparent display. This feature is
already available in higher end vehicles. This would be extremely useful for
detecting deer, pedestrians and even other vehicles in time to avoid an accident
(e.g., car crash). Another use for video cameras could keep track of lane lines,
and automatically adjust the steering to keep the car in the lane. Finally,
rear-facing cameras can give views of blind spots and displayed on a monitor
inside the car
Pedestrians’
Detection
Probably
the most important feature of a computer vision system for vehicle safety is to
detect pedestrians in a road and alert the driver for a crash. The number of
vehicle accidents in which pedestrians are involved are very high and contribute
significant amount of the vehicle accidents. Additionally, in this case human
lives are usually lost of severely damaged causing several social and economical
problems that are very important in a contemporary society.
Face
Detection
Face
detection systems can be considered as the first stage of pedestrian detection
system. It refers to the process of detecting image regions of human faces.
The
face detection algorithms can be applied by exploiting colour characteristics.
This is due to the fact that the space of the human face is distributed in a
very small region of colour space. Face recognition and localization should be
performed in this case without low memory and processing requirements, since the
alert and decision of the system should be performed automatically.
In
addition to face detection and localization, human body modelling should be
performed to detect the body f humans. Human body modelling can be applied with
respect to the human face location and thus it can increases the reliability and
robustness of a pedestrian detection system.
in-Color
Based Algorithms:
Various
methods and algorithms have been proposed in the literature over the years, for
human face detection, ranging from edge map projections to recent techniques
using generalized symmetric operators. Skin-based algorithms exploits the
properties of face color to perform the detection task.
The
efficiency of this method compared to other techniques is due to the fact that
·
the chrominance
components are directly available in compressed data
·
its computational
complexity is significantly low.
Shape
Constraints
Shape
constraints of the human faces are required to discard non-face regions, which
are detected as face by the skin-color modeling. In particular, the shape of
human faces is unique and consistent, the boundary of which can be approximated
by an ellipse, by connected arcs or by a rectangle. In our group, human faces
have been modeled using rectangles of certain aspect ratios.
Human
Body Detection
Another
important issue for a pedestrian detection is the human body detection tasks.
Followed human face detection, human body is localized by exploiting information
derived from the human face detection module. This is performed by human body
modeling. The model parameters, however, depend on the location of the human
face region. A probabilistic approach has been implemented by our group based on
the Gaussian pdfs.
Object
Detection
Apart
from the detection of pedestrians, object detection system should be also
accomplished. This way, stacks of woods, leaves, obstacles or other objects that
prevent the secure and secure traffic of vehicle are localized. Furthermore,
detection of humans or animals that may be cause accidents is within the
framework of the Cariman proposal.
Most
of the algorithms towards this direction exploit motion to detect passing or
crossing animal or human in the main streets. The same algorithms can be also
applied for vehicle detection.
Motion-Based
algorithms
Motion
information can be calculated in an image sequence by comparing the relative
position of the content of successive frames. Using appropriate models, complex
types of motion can be detection, such as object rotation, translation and/or
skewing. This can be performed for example by the use of affine transformations.
Having
detected the motion of the object in front of a vehicle, we can detect if the in
front object is crossing the street of goes ahead. Thus, we are able to have a
first signal for a possible crash. The camera system is also able to show us the
object location and thus according to the current position of the vehicle and
the speed and acceleration we can be able to give an alert for a possible crash.
Fast
algorithms for motion detection should be also implemented so that the decision
is performed on real time. This is very important especially in cases of such
systems, since the object may be presented abruptly and the decision about the
accident should be very urgent.
Classification-Based
algorithms
Other
types of algorithms for object detection are classification-based. These schemes
initially performs a classification of the color information and detect the
objects according to the abrupt changes of this information.
Object
Tracking
Tracking
of moving object is a very significant task for traffic safety using computer
vision tools. This is due to the fact that it enables fast object detection,
which is crucial for the a quick decision. Several tracking algorithms have been
presented in the literature in the recent year. These techniques are usually
boundary-based schemes, which start from an initial curve (providing either by
the user or by a segmentation algorithm) and then track the variation of object
boundaries by exploiting motion information. Although these approaches provide
good results for slow varying object boundaries, they are not suitable in case
that abrupt changes of the object boundary occur.
To
handle the aforementioned difficulty, a new approach is presented by our group,
which considers the tracking problem as a classification problem. In particular,
each video object is represented by a class and then, tracking is performed by
assigning each image region to the class (i.e., video object) of the highest
probability. To perform the classification task (i.e., object tracking), neural
networks are used in our case, due to their highly non-linear capabilities.
However, network weights cannot be considered constant through time, since video
objects of different scenes are characterized with different color or motion
properties. For this reason, we have developed an adaptable
neural network architecture.
Object
Type Classification
Another
important issue for traffic safety is the classification of the objects
detected. Classification permits automatic alerting of an emerging situation and
categorize the objects according to their conditions. Usually, classification is
performed using neural networks due to their high non-linear capabilities.
However, object type classification is a difficult task and requires advanced
object modeling, appropriate feature extraction and robust classifiers. In case
that constraints about the object type is imposed, the problem is simplified and
more accurate categorization can be accomplished.
Activity/
Event Recognition
Activity
recognition aims at identifying the events or movements of specific meaning.
Example include, recognition flying birds,
fire, detection of heavily raining or a unusual situations that may cause
accident. Activity recognition is performed by analyzing the characteristics of
a motion of a scene using appropriate models. Template matching approaches in a
motion feature space coupled with a technique for detecting and normalizing
periodic activities can be applied.
Non
Linear Multiscale Image analysis
Another
important issue towards traffic safety is to apply Multiscale Image Analysis.
This type of image processing has recently emerged as a useful tool for many
applications in image processing and computer vision society. Examples include, specific
shape object extraction, object
tracking as well as object modeling.
These features are useful for identification and of the objects.
Multiscale
image analysis is performed by applying an operator to image at different
"scales", resulting in a scale-space image decomposition. Representing
the image at different scales, several properties of the image content can be
extracted.
Linear
Scale - Space Analysis
Conventional
multiscale image analysis is performed by applying linear operators, such as
Guassian distributions at every image scale. However, linear operators cannot
efficiently describe the image visual content due to several reasons.
Particularly, a) they blur important regions of the image, such as the edges, b)
they diverse from the actual measured size of the scale and c) they cannot
provide a compact representation of the image content. To overcome the
aforementioned difficulties, non-linear filters can be used such as the
conventional morphological operators (dilation, erosion, opening and closing) as
well as the generalized ones.
Non-Linear
Scale-Space Analysis
Application
domains include the fields of geological, biomedical, and document image
analysis. Traditionally, the size distributions are formed by computing the
areas or volumes of standard morphological openings and closings (i.e.
compositions of Minkowski erosions and dilations) by convex structuring elements
(e.g. disks or lines) at multiple scales. However, this conventional approach
has weak points because the standard openings do not retain the contours of
image objects, cannot directly localize important image information such as the
area of its connected components, and are not oriented toward object-based
analysis. Furthermore, the information they provide about the image shape-size
content is spread among different scales and hence is not directly useful for
object-oriented analysis.
The
aforementioned difficulties are addressed by using size distributions and
corresponding size histograms based on generalized
multiscale openings. These can be formalized using the theory of image
operators on complete lattices. One class of generalized openings we use are the
reconstruction openings which can reconstruct whole objects (marked by some
seed) with extract preservation of their contour; in this reconstruction process
they simplify the original image by completely eliminating all objects inside
which the marker cannot fit. Another interesting class of generalized operators
are the area openings [9] which filter connected components of an image
according to their area. Both the reconstruction and the area openings are
connected operators; hence they are suitable for object-oriented size analysis.
Deformable
Models
A
deformable model is active in the sense that it can adopt itself to fit
the given data. It is a useful shape model because of its flexibility, and its
ability to both impose geometrical constraints on the shape and to integrate
local image evidences. There has been a substantial amount of research on
deformable models in recent years. These activities can be partitioned into two
classes:
·
free-form models, and
·
parametric models.
By
free-form deformable models, we mean that there is no global structure of the
template except for some general regularization constraints; the template is
constrained only by continuity and/or smoothness of the boundary. Such a
free-form model can be deformed to match salient image features like lines and
edges using potential fields (energy functions) produced by those features.
Since there is no global structure for the template, it can represent an
arbitrary shape as long as the regularization requirements are satisfied. On the
other hand, parametric deformable models control the deformations using a set of
parameters which are capable of encoding a specific characteristic shape and its
variations. This type of model is used when more specific shape information is
available, which can be described by a set of parameters. There are two ways to
parameterize the shape variation: (i) handcraft a parametric formula for the
curves (surfaces) in the shape template such that different shape instances can
be obtained using different parameter values; (ii) design a prototype for a
shape class, and then apply a parametric transformation on the prototype to
obtain different deformed templates.
Free-form
Deformation Models
Active
Contrours Active
contour is a power tool in the area of computer vision with many applications to
the traffic safety. The
active contour model, or snake, is defined as an energy-minimizing spline. The
snake’s energy depends on its shape and location within the image.
In this approach, an energy-minimizing contour, called a ``snake'', is
controlled by a combination of the following three forces or energies:
·
internal contour force
which enforces the smoothness,
·
image force which
attracts the contour to the desired features, and
·
external constraint
force.
Spline-based
Deformable Template Models
A spline-based template models more structured than a snake, because the
template is expressed as a linear combination of a set of basis functions, and
its shape is determined by the coefficients of the basis functions. However, the
linear combination can be arbitrary and does not usually encode a ``default''
shape as do the parametric models that are discussed later. In other words, the
model is less specific than specially designed templates for a specific shape
class. The choices of the spline basis can be quite broad including B-spline
basis, trigonometric basis, wavelets, etc.
Parametric
Deformation Models
A
parametric deformable template refers to the parametric shape model representing
the a priori knowledge about the structural properties of a class of
objects. By designing a global shape model, boundary gaps are easily bridged,
and overall consistency is more likely to be achieved. By parameterizing the
model, a compact description of the shape can be achieved. Parametric
deformation models are commonly used when some prior information of the
geometrical shape is available, which can be encoded using preferably, a small
number of parameters. There are two general ways to parameterize the shape class
and its variations:
Analytical
form-based parametric deformable models:
One can represent the shape as a collection of parameterized curves, i.e.,
parameterize the geometric shape directly. The template is represented by a set
of curves which is uniquely described by some parameters. The geometrical shape
of the template can be changed by using different values of the parameters.
Variations in the shape are determined by the distribution of the admissible
parameter values. This representation requires that the geometrical shapes be
well structured.
Prototype-based
parametric deformable models:
A so called ``standard'' or ``prototype'' or ``generic'' template is specified
to describe the ``most likely'' or ``average'' or ``characteristic'' shape of a
class of objects which has a global conforming structure and possibly individual
deviations. Each instance of the shape class is derived from the ``prototype''
via a parametric mapping. The use of different parameter values again gives rise
to different shapes. Variations in the shape are also determined by the
distribution of the admissible parameter values of the mapping.
In
both the parametric models mentioned above, the deformable templates interact
with the image features dynamically by adjusting the parameters according to the
image forces. Similar to the active contour approach, an objective function
which is a weighted sum of an internal energy term and an external energy term
is used to quantify how well a deformed template matches the objects in the
given image. Recall that in the active contour approach, the internal energy, in
terms of the stretchness and the elasticity of the spline, actually imposes a
rather general and weak a priori distribution on the contour model,
i.e., the contour should be smooth and compact. In the parametric deformable
template approaches, where the a priori shape preferences are
explicitly encoded by the parameters, a similar internal energy term is defined
based on the constraints and interactions on the geometrical structures. For
example, it can be defined to penalize the deviation of the deformed template
from the ``expected'' shape. The external energy term, which pertains to the
fidelity of the deformed template to the input image, is introduced so that the
template deforms according to the desired goal. It is perceived that the
internal energy corresponds to a geometric measure of the fitness, and the
external energy corresponds to an image fidelity measure of fitness. The two
fitness measures are combined to give an overall measure of fitness,
appropriately weighting both the prior knowledge and the image data. The set of
parameters which optimizes the objective function gives a description of the
detected or matched shape. The value of the objective function quantifies the
plausibility of the detection.
Guided
by Computer Vision
A
different approach to computer driving has been to employ video cameras and
complicated computer vision algorithms to keep the vehicle within the lane. The
lines of the road provide the means of computer navigation and rates of change
of curvature in the on coming road let the computer know how to adjust the
steering. Though this approach does not require an investment in infrastructure
(i.e. embedded transmitters on the highways), it unleashes a host of other
problems.
The
major road blocks encountered by researchers in this venture are:
·
markings differ
between roadways, some having no lines at all;
·
complications
arise in distinguishing turning lanes, and again markings are inconsistent.
·
weather
conditions such as snow would obscure lines completely.
Decision
Systems
Decision
systems for traffic safety are very important. This is due to the fact that
these systems combines data stem from different sensors so as to reach a correct
decision about the traffic safety. Decision systems include data interpretation
tools, data simplification tools, data clustering and classification and data
abstraction.
Data
Interpretation
The
goal of this system is to transform data into a form that can be interpreted so
as to give alert only in situations that a need is required. Advanced signal
processing tools and algorithms can be applied for data interpretation. Modeling
of the extracted data is also required.
Data
Simplification
Data
Mining
Data
mining can be used for accelerating the processing of the data extracted. This
is very important for traffic safety since fast decision is the main factor to
avoid an accident. Generally, the purpose of a data mining module is
a..
to organise and classify information relative to a patients medical record
b.
to extract information from patterns or data series
c.
to associate extracted information with symptoms.
Incorporation
of various data mining algorithms is needed in order to cope with all possible
data mining problems, since the nature of the data mining problem often suggests
a certain class of data mining technique or method to be an appropriate
solution. The data mining problems that will be subjected to research are the
following:
Classification
– This problem involves the need to find rules that can partition the data
into disjoint groups. Classification involves supervised data mining tools in
which the user is heavily involved in the definition of the different groups and
the specification of the rules that can be used to determine to which group a
data item belongs. In addition, unsupervised algorithms can be adopted, which
separates the classes based on the variation of the data elements. Examples of
such tools include decision trees and rule-based techniques. Example-based data
mining methods that will be investigated during the project are the nearest
neighbour classification, regression algorithms and case-based reasoning.
Association
–
The association data mining problem involves finding all of the rules (or at
least a critical subset of rules) for which a particular data attribute is
either a consequence or an antecedent. This type of problem is very common in
marketing data mining problems but is also of interest to health care
professionals who are looking for relationships between diseases and life-styles
or demographics or between survival rates and treatments, for example.
Association problems are similar to rule-based methods, but in addition they
typically have confidence factors associated with each rule. For this reason, an
example of such a technique is a probabilistic graphical dependency model (with
a Bayesian component). Often association-type data mining techniques are
employed to help strengthen arguments concerning whether or not to include or
eliminate candidate rules from a knowledge model.
Non-Linear
Decision Tools-Artificial Intelligence
More
accurate decision can be reached using artificial neural networks. Neural
networks are systems composed of a large number of basic elements arranged in
layers and that are highly interconnected. The structure consists of many inputs
and outputs. The inputs corresponds to the data used to reach an decision while
the outputs contains the decision made.
Due
to their highly non-linear capabilities they have been extended used in a
variety of classification and decision support systems with a great success
compared to other structure. In addition, they are general architecture that can
be taught to interpret several aspects with high efficiency. Thus, they are not
application-oriented.
Several
neural networks structures exist and can be used for classification purposes.
Probably, the most common is the feedfoward neural network architecture. In
these type of networks, the inputs are related to the outputs through different
layers and no backward interconnection is permitted. The backpropagation
training algorithm is the most common used to estimate the weights
of a neural network. Several modifications of the algorithm has been
applied.
Other
accurate neural network-based classification models are the Learning Vector
Quantization schemes. These types of networks are oriented for non-linear
classification. In addition, modular of hierarchical netowrks can be also used
or bayesian networks where no training is required.
However,
probably the most important issue when designing and training artificial neural
networks in real life applications is network generalization, i.e., the network
performance to data outside the training set. Many significant results have been
derived during the last few years regarding generalization of neural networks
when tested outside their training environment. Examples include algorithms for
adaptive creation of the network architecture during training, such as pruning
or constructive techniques, or theoretical aspects of network generalization,
such as the VC dimension. Specific results and mathematical formulations
regarding error bounds and overtraining issues have been obtained when
considering cases with known probability distributions of the data.
For
the problem of traffic safety, network generalisation is very crucial since the
network performance should remain accurate in all conditions and environments
regardless of the training samples. Despite, however, the achievements obtained,
most real life applications do not obey some specific probability distribution
and may significantly differ from one case to another mainly due to changes of
their environment. That is why straightforward application of trained networks,
to data outside the training set, is not always adequate for solving image
recognition, classification or detection problems.
In
our group, an adaptable neural network architecture has been implemented able to
automatically adjust its performance to the current environmental conditions.
The architecture consists of a an retraining
set construction module, a retraining
algorithm and a decision mechanism
which detects the time instances that a new network retraining is required.
Fuzzy
Classifications
Fuzzy
classification can be used for increasing the performance of data clustering.
Many spatial phenomena can not be represented properly with conventional
deterministic classification techniques. Instead, the elaboration of a fuzzy set
approach with regards to mapping can be of particular advantage for representing
complex spatial systems. The method can generally be used for mapping any
multidimensional spatial data.
Expert
Systems
Within
the last ten years, artificial intelligence-based computer programs called expert
systems have received a great deal of attention. The reason for all the
attention is that these programs have been used to solve an impressive array of
problems in a variety of fields. Well-known examples include computer system
design, locomotive repair, and gene cloning
An
expert system stores the knowledge of one or more human experts in a particular
field. The field is called a domain. The experts are called domain experts. A
user presents the expert system with the specifics of a problem within the
domain. The system applies its stored knowledge to solve the problem.
Expert
systems are very important for traffic safety since they are able to efficiently
combine different conditions and data extracted from different sensors so as to
derive an accurate decision.