ICS Dr. G. Roscher GmbH
Real-time Recognition of Noisy Signals - from Signal to Knowledge
For over 15 years Dr. G. Roscher has been developing a method for the real-time recognition of signals in the time domain - in contrast to the established frequency domain methods such as the Discrete Fourier Transform (DFT). This new method is based on the “old” peak measurement and evaluates each event in the signal as:
- extreme amplitude,
- extreme slope (optional),
- and as an further option each extreme curvature.
Each event is transformed into a data structure named Virtual Source (in German: Virtuelle Quelle – VQ). The result of this transformation is the description of the signal as a sequence of VQs. Further steps build up a hierarchical system of chained lists of VQs, named
- Cycles and
description of the signal can be easily manipulated by mathematical methods and
can be easily recognized. Previous researchers have not appreciated the
sophisticated performance in the time domain of the hard-wired parallel
processing human visual or auditory system and brain. All the established
groups have used frequency domain methods! These methods are approximations
which lack the accuracy and performance of the human recognition system.
This high performance requires the application of time domain methods for signal recognition of the highest accuracy, which employs the latest technologies in the fields of Data Base (DBS) and Knowledge Management Systems (KMS). Each incoming signal is stored in the DBS. These signals are transformed in the VQs, segmented and indexed by the time and by the segment number. The classification is achieved using KMS for the acquisition of personal knowledge in direct communication between the user and the KMS.
the Human Visual System, cones and rods are connected by neurons to the brain.
Cones, rods and connecting neurons are powerful systems; they carry out complex
and complicated information processes for signal analysis and recognition in the
time domain at different levels.
These information processes are modeled by the method of Virtual Sources VQs.
Albert Einstein said
“We should make
things as simple as possible, but not simpler”.
VQs on different levels are characterized in general by:
a [, aR, aG, aB]
p [, pR, pG, pB]
w [, wR, wG, wB]
k [, kR, kG, kB]
localization of the channel or the source
[x, y [, z]]
of pointers to VQs of the same and lower levels.
The recognition is performed by using the following methods:
- Fuzzy-logic for pattern recognition,
- Neural networks for feature extraction and classification,
- Evolutionary algorithms for classification,
- Dynamic lists for internal management of the signal description in real-time,
- Data Base System for external management of the signal description.
- Knowledge Management System for acquisition of expert knowledge.
Special heuristic algorithms for solutions without algorithms.
classification of the evolutionary algorithm is described for this simple
classification at the first level
led in a fuzzy manner to two classes: Class1, named “VQClass black to
white” and Class2, named “VQClass white to black”. In the
second level, all VQs generated at the same time are collected and used to build
the two lines described by the VQs of level 1. That will be the recognized
object named “VQ White line on the blackboard”. Some times later,
the same object appears to be translated in a defined direction. The description
of this object is in a fuzzy manner the same as the “VQ White line on the
evolutionary algorithm for automatic classification worked in the following way:
The first recognised object is stored in the so called garbage class. Each following object is compared with the members of the garbage class. If the new object matches with one member of the garbage class, to a defined degree they found a new class. These two objects are the parents of the new class and will never be eliminated from the class description. Each new object which matches with the members of this class is added to the class description, up to tmax members in the class description. tmax will be greater than two. If tmax+1 members are in the class description, one member is eliminated by a heuristic algorithm. If one object does not match with an established class description, it is checked against members of the garbage class. If it matches with one member in the garbage class, the two objects are the parents of a new class, else the new incoming object is stored in the garbage class.
user can evaluate the VQs as description of real objects and can name these
objects or classes by using interactive methods in a graphical user interface.
The notion of naming objects which has a computer aided description by the user
builds a new type of knowledge management system. This is demonstrated by the
application of the classification and recognition of single channel ECG-signals
in Fig. 2 by using the wireless portable PhysioCord
and the high performance stationary ECG-System HeartScope.
Fig.2: The ECG as example for automatic classification and recognition
The application of the method is
performed for the single channel ECG using a
high sampling accuracy (sampling rate: 512 samples / sec). In
Fig. 2 the classification of single channel ECG is presented (only the first red
marked channel EKG2 is analysed). The red numbers
signify the beat number of the 24 hour ECG, going from 70021
to 70037 in Fig. 2. The blue
numbers are the Inter Beat Interval (IBI) and go from 360
in minimum to 493 in this example. This high
variation is generated by the pathological heart beat 70022.
The green numbers are the automatic generated
classes and go from 1 = garbage class to class
37 in this example.
Class 2 (Klasse
2, Anzahl = number = 35064 beats) and Class 3 (Klasse
3, Anzahl = number = 41999 beats) are the normal heartbeats, differentiated by
the slope of R and S. The representation of
these normal heart-beats is quite different from
the representation presented in the literature. The high flexibility of
feature extraction, recognition and evolutionary algorithm for classification
opened this innovative way.
Class 15 are
pathological heartbeats, 11 in one day. When
Heartbeat 6161 appears first, it doesn’t
match with one of the existent classes and is stored in the garbage class. The
Heartbeat 9408 matches first with heartbeat 6161,
and they form class 15. Only 5 heartbeats are
used as Templates for the class description (tmax =
5). Heartbeat 70022
is classified in class 15, and is a
member of the class description and is presented in the ongoing ECG, in ECG-channel
The experienced physician can evaluate the automatic generated classes, can name the classes, can delete non-significant classes and can introduce his knowledge to the automatically generated signal description.
EEG system BrainScope
a special amplifier system for high quality signal detection in open field
conditions during communicative situations. A high performance computer system
which is capable of processing the huge amounts of data produced by a multi
channel EEG record to gain information in real-time has also been developed.
Algorithms for recognition of events in single channels are implemented in the
first level of the computer system. High performance 3D image processing
algorithms are used in the second level, interpreting the sampled values of each
channel as pixels of the image, from 256 to 2000 times per second. This method
describes the EEG activity as sequences of VQs with parameters of amplitude,
time and space.
The network of two or more Personal Computers (PCs) is co-ordinated through the computer system for presentation of EEG activity and control. Multi-media approaches to the application of psychological tests are possible through the user interface including tests in media of sound, words, pictures and moving pictures. These tests can be arranged and carried out in computer controlled sequences and modified by user interactions. Tools are also provided to allow the user to create his own tests. These methods are integrated into the powerful Graphic User Interface and use a Data Base System. Incorporated into this User Interface are state of the art EEGSYS algorithms from the NIMH (Washington / USA) for mappings, FFT, etc.
The BrainScope demonstrates the impacts and applications of the new strategy for EEG investigation in communicative situations between:
- patient and physician for subjective evaluation,
- patient and information technology for stimulation and acquisition of signals and reactions,
- physician and information technology for quantitative analysis of signals and reactions.
major advantage of this new strategy is that the three processes can be carried
optimises the capacity of humans to interpret information with the capability of
modern information technology to
manipulate and process data. It therefore requires use by an experienced and
trained operator who can make accurate observations during the process of an
investigation. The user can, for example, click on a significant EEG pattern
(this makes it a further recognisable phenomenon through fuzzy logic) and
correlate it with his own observations. The computer system recognises this EEG
activity, i.e. it interprets this as a possible description of the state of the
brain, sets a defined stimulus and recognises and evaluates the Event Related
Potential (ERP) immediately.
Fig. 3: The EEG for a complex, 3D signal
ERP are EEG-changes, related to a particular event (e. g. acoustic or visual stimulus or motor reactions) and give hints to the underlying information process.
Fig 3 the recognition of Event Related Potentials (ERPs) in the EEG is presented
as N1, P2, N2 and P3 components in a single trial without averaging. In the
right part the ongoing signal is presented. The red line marks the start of the stimulus
(Reiz = stimulus) and the blue = reaction.
The generated ERPs are marked by coloured lines:
magenta = N1,
– in the EEG, the negative signal is above!
grey = N2,
the left the sequence of coloured maps is presented using the powerful
3D-Mapping of the NIMH / Washington. Normally, each sample generates one map.
Because of space restrictions, only significant maps are shown in Figure 3. The
white crosses are the symbols for the VQs and describe the evolution of the
appearance from the start up to the extreme value of the amplitude. These ERPs
can be seen in the many channels EEG in the right side of in Fig. 3 by the
vertical lines. The N2 has two components: one
earlier component in the central region and one in the frontal region, 16
milliseconds later. These VQs are the compressed description of the sequence of
maps on the left side of Fig. 3. The description of the named objects in the
complex 3D-signal can be easily recognized and manipulated by data base
mechanisms and statistics.
The real localisation of the EEG-activity in the brain is not possible by mathematical methods (inverse problem). The VQ is a simple evaluation in the computer model, using the centre of gravity of all clustered potentials. The basic hypothesis for this method is as follows:
The same electrical activity in the brain translates consistently to
the same electrical activity
detected by the electrodes on top of the head and therefore creates
the same Virtual Sources.
In this way the computer builds up the ongoing EEG as a sequence of VQs.
The user can select important VQs with the mouse and define templates for the recognition of the VQ in the ongoing complex signal. These predefined templates are chosen from either the EEG display or the ERP display (in the presented example: N1, P2, N2 and P3). The ongoing signal display can be examined stepwise by locating a line cursor and continuously clicking the mouse, each VQ of the current click can be figured and displayed in a list box. The user can name the VQs, can manipulate the proposed parameters of the description of the VQs and store this description in the Data Base System. There is a user-friendly way to train the system to recognise specialised events in the complex signal.
However, the true value of the system becomes evident when it is trained to detect and estimate latency, virtual localisation and amplitude of the complex signal. With the taught high performance computer system, the patterns can be recognised in milliseconds. The templates have to be selected to best represent the pattern which is intended to be recognised.
If a priori defined EEG-activity occurred, described by a stored sequence of VQs during the EEG reading, the stimulus was given to the patient. What happens is that the taught high performance computer system recognises the sequence of VQs and then starts a predefined action with a defined delay. We name this feature of the system: "stimulus, triggered by the state of the system".
The system can be used in three modes:
Learning mode: the
methods of recognition and classification works with imprinted methods for
feature extraction and classification. New classes are built by the algorithms.
- Teaching mode: the qualified user evaluates the classes or specialised features in the
signal. This description is understandable for the user and the computer.
- Evaluation mode: the evaluated classes or specialised features are the basis for
recognition and classification in autonomous applications.
are possible for noisy signals as
1. s = f(t)
s = f(x, y, t)
s = f(x, y, z, t)
The advantage of the computer system is the evaluation of each pattern of the signal over a long duration without loss of attention and with a short reaction time (BrainScope <= 50 ms; human being>=200ms).