|
ICS Dr. G. Roscher GmbH |
SoftComputing |
Real-time Recognition of Noisy Signals - from Signal to Knowledge
For over 25 years Dr. G. Roscher has been developing a method for the real-time recognition of signals
Each event is transformed into a data structure named Virtual Source (in German: Virtuelle Quelle – VQ). The result of this transformation is the description of the signal as a sequence of VQs. Further steps build up a hierarchical system of chained lists of VQs, named
- SuperPeaks,
- Cycles and
- Classes.
This description of the signal can be easily manipulated by mathematical methods and can be easily recognized. Previous researchers have not appreciated the sophisticated performance in the time domain of the hard-wired parallel processing human visual or auditory system and brain. All the established groups have used frequency domain methods! These methods are approximations which lack the accuracy and performance of the human recognition system.
This high performance requires the application of time domain methods for signal recognition of the highest accuracy, which employs the latest technologies in the fields of Data Base (DBS) and Knowledge Management Systems (KMS). Each incoming signal is stored in the DBS. These signals are transformed in the VQs, segmented and indexed by the time and by the segment number. The classification is achieved using KMS for the acquisition of personal knowledge in direct communication between the user and the KMS.
In the Human Visual System, cones and rods are connected by neurons to the brain. Cones, rods and connecting neurons are powerful systems; they carry out complex and complicated information processes for signal analysis and recognition in the time domain at different levels. These information processes are modeled by the method of Virtual Sources VQs.
As Albert Einstein said
“We should make things as simple as possible, but not simpler”.
The recognition is performed by using the following methods:
- Fuzzy-logic for pattern recognition,
- Neural networks for feature extraction and classification,
- Evolutionary algorithms for classification,
- Dynamic lists for internal management of the signal description in real-time,
- Data Base System for external management of the signal description.
- Knowledge Management System for acquisition of expert knowledge.
- Special heuristic algorithms for solutions without algorithms.
The classification of the evolutionary algorithm is described for this simple example:
The classification at the first level led in a fuzzy manner to two classes: Class1, named “VQClass black to white” and Class2, named “VQClass white to black”. In the second level, all VQs generated at the same time are collected and used to build the two lines described by the VQs of level 1. That will be the recognized object named “VQ White line on the blackboard”. Some times later, the same object appears to be translated in a defined direction. The description of this object is in a fuzzy manner the same as the “VQ White line on the blackboard”.
The evolutionary algorithm for automatic classification worked in the following way:
The first recognised object is stored in the so called garbage class. Each following object is compared with the members of the garbage class. If the new object matches with one member of the garbage class, to a defined degree they found a new class. These two objects are the parents of the new class and will never be eliminated from the class description. Each new object which matches with the members of this class is added to the class description, up to tmax members in the class description. tmax will be greater than two. If tmax+1 members are in the class description, one member is eliminated by a heuristic algorithm. If one object does not match with an established class description, it is checked against members of the garbage class. If it matches with one member in the garbage class, the two objects are the parents of a new class, else the new incoming object is stored in the garbage class.
The user can evaluate the VQs as description of real objects and can name these objects or classes by using interactive methods in a graphical user interface. The notion of naming objects which has a computer aided description by the user builds a new type of knowledge management system. This is demonstrated by the application of the classification and recognition of single channel ECG-signals in Fig. 2 by using the wireless portable PhysioCord and the high performance stationary ECG-System HeartScope.
Fig.2: The ECG as example for automatic classification and recognition
The application of the method is performed for the single channel ECG using a high sampling accuracy (sampling rate: 512 samples / sec). In Fig. 2 the classification of single channel ECG is presented (only the first red marked channel EKG2 is analysed). The red numbers signify the beat number of the 24 hour ECG, going from 70021 to 70037 in Fig. 2. The blue numbers are the Inter Beat Interval (IBI) and go from 360 in minimum to 493 in this example. This high variation is generated by the pathological heart beat 70022. The green numbers are the automatic generated classes and go from 1 = garbage class to class 37 in this example.
Class 2 (Klasse 2, Anzahl = number = 35064 beats) and Class 3 (Klasse 3, Anzahl = number = 41999 beats) are the normal heartbeats, differentiated by the slope of R and S. The representation of these normal heart-beats is quite different from the representation presented in the literature. The high flexibility of feature extraction, recognition and evolutionary algorithm for classification opened this innovative way.
Class 15 are pathological heartbeats, 11 in one day. When Heartbeat 6161 appears first, it doesn’t match with one of the existent classes and is stored in the garbage class. The Heartbeat 9408 matches first with heartbeat 6161, and they form class 15. Only 5 heartbeats are used as Templates for the class description (tmax = 5). Heartbeat 70022 is classified in class 15, and is a member of the class description and is presented in the ongoing ECG, in ECG-channel EKG2.
The experienced physician can evaluate the automatic generated classes, can name the classes, can delete non-significant classes and can introduce his knowledge to the automatically generated signal description.
The EEG system BrainScope consists of a special amplifier system for high quality signal detection in open field conditions during communicative situations. A high performance computer system which is capable of processing the huge amounts of data produced by a multi channel EEG record to gain information in real-time has also been developed. Algorithms for recognition of events in single channels are implemented in the first level of the computer system. High performance 3D image processing algorithms are used in the second level, interpreting the sampled values of each channel as pixels of the image, from 256 to 2000 times per second. This method describes the EEG activity as sequences of VQs with parameters of amplitude, time and space.
The network of two or more Personal Computers (PCs) is co-ordinated through the computer system for presentation of EEG activity and control. Multi-media approaches to the application of psychological tests are possible through the user interface including tests in media of sound, words, pictures and moving pictures. These tests can be arranged and carried out in computer controlled sequences and modified by user interactions. Tools are also provided to allow the user to create his own tests. These methods are integrated into the powerful Graphic User Interface and use a Data Base System. Incorporated into this User Interface are state of the art EEGSYS algorithms from the NIMH (Washington / USA) for mappings, FFT, etc.
The BrainScope demonstrates the impacts and applications of the new strategy for EEG investigation in communicative situations between:
- patient and physician for subjective evaluation,
- patient and information technology for stimulation and acquisition of signals and reactions,
- physician and information technology for quantitative analysis of signals and reactions.
The major advantage of this new strategy is that the three processes can be carried out
in real-time.
It optimises the capacity of humans to interpret information with the capability of modern information technology to manipulate and process data. It therefore requires use by an experienced and trained operator who can make accurate observations during the process of an investigation. The user can, for example, click on a significant EEG pattern (this makes it a further recognisable phenomenon through fuzzy logic) and correlate it with his own observations. The computer system recognises this EEG activity, i.e. it interprets this as a possible description of the state of the brain, sets a defined stimulus and recognises and evaluates the Event Related Potential (ERP) immediately.
Fig. 3: The EEG for a complex, 3D signal
ERP are EEG-changes, related to a particular event (e. g. acoustic or visual stimulus or motor reactions) and give hints to the underlying information process.
In Fig 3 the recognition of Event Related Potentials (ERPs) in the EEG is presented as N1, P2, N2 and P3 components in a single trial without averaging. In the right part the ongoing signal is presented. The red line marks the start of the stimulus (Reiz = stimulus) and the blue = reaction. The generated ERPs are marked by coloured lines:
magenta = N1, – in the EEG, the negative signal is above!
brown = P2,
grey = N2,
black =P3
On the left the sequence of coloured maps is presented using the powerful 3D-Mapping of the NIMH / Washington. Normally, each sample generates one map. Because of space restrictions, only significant maps are shown in Figure 3. The white crosses are the symbols for the VQs and describe the evolution of the appearance from the start up to the extreme value of the amplitude. These ERPs can be seen in the many channels EEG in the right side of in Fig. 3 by the vertical lines. The N2 has two components: one earlier component in the central region and one in the frontal region, 16 milliseconds later. These VQs are the compressed description of the sequence of maps on the left side of Fig. 3. The description of the named objects in the complex 3D-signal can be easily recognized and manipulated by data base mechanisms and statistics.
The real localisation of the EEG-activity in the brain is not possible by mathematical methods (inverse problem). The VQ is a simple evaluation in the computer model, using the centre of gravity of all clustered potentials. The basic hypothesis for this method is as follows:
The same electrical activity in the brain translates consistently to
the same electrical activity
detected by the electrodes on top of the head and therefore creates
the same Virtual Sources.
In this way the computer builds up the ongoing EEG as a sequence of VQs.
The user can select important VQs with the mouse and define templates for the recognition of the VQ in the ongoing complex signal. These predefined templates are chosen from either the EEG display or the ERP display (in the presented example: N1, P2, N2 and P3). The ongoing signal display can be examined stepwise by locating a line cursor and continuously clicking the mouse, each VQ of the current click can be figured and displayed in a list box. The user can name the VQs, can manipulate the proposed parameters of the description of the VQs and store this description in the Data Base System. There is a user-friendly way to train the system to recognise specialised events in the complex signal.
However, the true value of the system becomes evident when it is trained to detect and estimate latency, virtual localisation and amplitude of the complex signal. With the taught high performance computer system, the patterns can be recognised in milliseconds. The templates have to be selected to best represent the pattern which is intended to be recognised.
If a priori defined EEG-activity occurred, described by a stored sequence of VQs during the EEG reading, the stimulus was given to the patient. What happens is that the taught high performance computer system recognises the sequence of VQs and then starts a predefined action with a defined delay. We name this feature of the system: "stimulus, triggered by the state of the system".
The system can be used in three modes:
- Learning mode: the methods of recognition and classification works with imprinted methods for feature extraction and classification. New classes are built by the algorithms.
- Teaching mode: the qualified user evaluates the classes or specialised features in the signal. This description is understandable for the user and the computer.
- Evaluation mode: the evaluated classes or specialised features are the basis for recognition and classification in autonomous applications.
Applications are possible for noisy signals as
1. s = f(t)
2. s = f(x, y, t)
3. s = f(x, y, z, t)
The advantage of the computer system is the evaluation of each pattern of the signal over a long duration without loss of attention and with a short reaction time (BrainScope <= 50 ms; human being>=200ms).