ICS Dr. G. Roscher GmbH
MikroSytemTechnik für Life Sciences
Landesinitiative Mikrosystemtechnik Niedersachsen, Hannover, 9. Oktober 2003
zur Echtzeiterkennung des EKG
Methods and Microsystems for the Real-Time Recognition of noisy Signals
Dipl.-Math. Günther Roscher
ICS Dr. G. Roscher GmbH
– 39326 Klein Ammensleben
+ 49 39202 5212-6, Fax: -8
EEG system BrainScope consists of a special amplifier system for high quality
signal detection in open field conditions during communicative situations. A
high performance computer system which is capable of processing the huge amounts
of data produced by a multi channel EEG record to gain information in real-time
has also been developed. Algorithms for the recognition of events in single
channels are implemented in the first level of the computer system. High
performance 3D image processing algorithms are used in the second level,
interpreting the sampled values of each channel as pixels of the image, from 256
to 2000 times per second.
network of two or more Personal Computers (PCs) is co-ordinated through the
computer system for presentation of EEG activity and control. Multi-media
approaches to the application of psychological tests are possible through the user interface including
tests in media of sound, words, pictures and moving pictures as videos. These
tests can be arranged and carried out in computer-controlled sequences and
modified by user interactions. Tools are also provided to allow the user to
create his own tests. These methods are integrated into the powerful Graphic
User Interface and use a Data Base & Knowledge Management System.
Incorporated into this User Interface are state of the art EEGSYS algorithms
from the NIMH (Washington / USA) for mappings, FFT, etc.
BrainScope demonstrates the impacts and applications of the new strategy for EEG
investigation in communicative situations between:
patient and physician for subjective evaluation,
patient and information technology for stimulation and acquisition of signals
physician and information technology for quantitative analysis of signals and
major advantage of this new strategy is that the three processes can be carried
optimises the capacity of humans to interpret information with the capability of
modern information technology to manipulate and process data. It therefore
requires use by an experienced and trained user who can make accurate
observations during the process of an investigation. The user can, for example,
click on a significant EEG pattern (this makes it a further recognisable
phenomenon through Fuzzy Logic) and correlate it with his own observations. The
computer system recognises this EEG activity, i.e. it interprets this as a
possible description of the state of the brain, sets a defined stimulus and
recognises and evaluates the Event Related Potential (ERP) immediately.
are EEG-changes, related to a particular event (e.g. acoustic or visual stimulus
or motor reactions) and give hints to the underlying information process.
In Fig 5 the recognition of Event Related Potentials (ERPs)
in the EEG are presented as N1, P2, N2 and P3 components in a single trial
In the middle part is presented the averaged ERP, using
methods of on-line averaging of the ongoing test. The ongoing signal is
presented in the right part.
You can recognise the equivalences and differences between
the averaged ERP and the activities in the single trial. The red line marks the
start of the stimulus (Reiz = stimulus) and the blue
= reaction. The generated ERPs are marked by coloured lines:
magenta = N1,
– in the EEG, the negative signal is above!
brown = P2,
grey = N2,
On the left coloured maps are presented using the powerful
3D-Mapping of the NIMH / Washington. Normally, each sample generates one map.
Because of space restrictions in the Figure, only the significant 3D maps for
the P3 component are shown. The white crosses are the symbols for the
ERP-component and describe the evolution of the appearance from the start up to
the extreme value of the amplitude. These ERPs can be seen in the many channels
EEG in the right side of in Fig. 5 by the vertical line for the P3-component.
The description of the P3-component in the complex 3D-signal can be easily
recognized and manipulated by data base mechanisms and statistics.
user can select such important or interesting activities with the mouse and
define templates for the recognition of the activities in the ongoing complex
signal. These predefined templates are chosen from either the EEG display or the
ERP display. The user can name this object, e.g. P3 and store the features in the
ongoing signal display or the on-line averaged ERP can be examined stepwise by
locating a line cursor and continuously clicking the mouse, each activities of
the current click can be figured and displayed in a list box, the Select Window.
In this Select Window are presented the numerical values of the marked
activities. The user can name the selected activities as the P3 in the Fig. 5,
can manipulate the proposed parameters of the description by using methods of
Fuzzy Logic and can store this description in the DB&KMS. There is a
user-friendly way to train the system to recognise specialised events as the
ERPs in the complex signal.
the true value of the system becomes evident when it is trained to detect and
estimate the activities of the complex signal. With the taught high performance
computer system, the patterns can be recognised in milliseconds. The templates
have to be selected to best represent the pattern which is intended to be
a priori defined EEG-activity occurred, described by a stored sequence of
activities during the EEG reading, the stimulus was given to the patient. What
happens is that the taught high performance computer system recognises the
sequence of activities and then starts a predefined action with a defined delay
in the range of 50 milliseconds. We name this feature of the system: "stimulus,
triggered by the state of the system".
one wishes to recognise the function of the brain, one must use the most
powerful tools: human intelligence, especially the natural language in
communicative situations, supported by high performance computer systems, worked
in real-time. "Moreover, humans can describe in words what they have just
seen. They can also tell us what they are imagining or what they have just
dreamt. It is almost impossible to get such information from a monkey." [Crick,
the course of a discussion, an experienced doctor or psychologist is able to
obtain a very exact subjective diagnosis of a patient's condition;
computer-aided psychological tests serve to strengthen these subjective
diagnoses. During the course of
these tests, it is often useful for the researcher to continue to examine the
behaviour of the experimental subject, to vary the test based on the subject's
personality and previously achieved test results, and to sharpen his
professional opinion of the subject's condition.
F. Crick described the problem: Nobody knew the context between EEG-activity and
the underlying information process [Crick, p 111].
communicative situation between patient, physician and computer system in this
here described innovative way build the technical support for a new quality of
brain research, gives a solution to Cricks problem.
is a civilization disease with a high risk of fatality especially in industrial
regions like Europe. At the same time, such industrialized regions exhibit
frequently high unemployment patterns coupled with stress related syndromes on
the persons considered which might contribute to such diseases. This is the
motivation for innovative product development using national and international
co-operation to overcome unemployment on the one hand and to recognize persons
under increased health risk on the other.
real-time recognition of the electrical activities of the heart, known as the
electrocardiogram (ECG), is a unique feature of the developed system which uses
powerful information technologies as its technical basis for continuous
monitoring of patients at risk, and recognizes critical situations in real-time
with highest accuracy. In addition, state-of-the-art communication technologies
are also used for the transmission of relevant data to present the risk
situation of the patient to a qualified physician for making decisions. The
scientific background for this work is related to the project "Methods of
Nonlinear Dynamic for analysis of the ECG, for risk stratification and therapy
assessment for heart patients", supported by the German Ministry of
Education and Research (Grant 13N7129) and active R&D-activities in close
cooperation with pharmacological enterprises.
ECG system HeartScope based on the
technology and methods of the BrainScope.
logic and methods of AI are used to define and recognise activities as
QRS-complexes and heart beats in real-time.
The innovative methods realise a description
of the signal, can be easily manipulated by mathematical methods and can be
common recognition system recognizes the ECG and builds classes automatically.
The experienced physician evaluates the ECG and the classes, marks significant
and interesting signal structures in the ECG and in the classes as well as using
a powerful Graphical User Interface. This process generates the formal
description of the signal in detail, ECG-substructures (P, Q, R, S, T) up to
heartbeats and classes. The critical pattern may be
failing to identify proper ECG’s, e.g. by cardiac arrest, appearance of
iterative ECG’s with an elevated S-T, P-Q segments, e.g. acute myocardial
infarction (AMI), or grouped ECG’s with desperate elements, e.g. torsade de
points. All of these critical patterns can be adapted individually to properly
fit the properties of the individual patients. A permanent monitoring of
patients at risks, e.g. after an AMI or at the beginning, during changes or
respective dosage adaptation of a specific drug treatment (anti-arrhythmica)
gives new insight in these processes. Each of these patterns with its
descriptive details is stored in the DB & KMS permitting further analysis.
If a pattern does not match with previously described patterns a new class of
pattern will be opened. In this way several classes of patterns of usually
occurring ECG’s arise and can be offered for matching. Note that patterns
indicating harmful situation or being critical for surviving can be introduced
in this pattern recognition program by the physician.
The application of the method is demonstrated for the
single channel ECG using a sampling rate of 512 samples / sec. The
classification of single channel ECG is presented in Fig. 7 (only the first red
marked channel EKG2 is presented).
The red numbers signify the beat number of
the 24-hour ECG, going from 70021 to 70037
in Fig. 7. The blue numbers are the markers
for the Inter Beat Interval (IBI) and are going from 360
in minimum to 493 in this example. This high
variation is generated by the pathological heartbeat 70022.
The green numbers are
the automatic generated classes and are going from 1
= garbage class to class 37
in this example.
(Klasse 2, Anzahl = number = 35064 beats) and Class
3 (Klasse 3, Anzahl = number = 41999 beats) are the normal heartbeats,
differentiated by the slope of R and S. The representation of these normal
heartbeats of the ECG is quite different from the representation presented in
the literature. The high flexibility of feature extraction, recognition and
evolutionary algorithm for classification opened this innovative and flexible
represents pathological heartbeats, 11 in one day. When Heartbeat 6161
appears first, it doesn’t match with one of the existent classes and is stored
in the garbage class. The Heartbeat 9408 matches
at first with heartbeat 6161, and they form the class
15. Only 5 Heartbeats are used as Templates for the class description (tmax
= 5). Heartbeat 70022 is
classified in class 15, and is a member of
the class description and is presented in the ongoing ECG, in ECG-channel EKG2
in Fig. 7.
The experienced physician can evaluate the automatic
generated classes, can name the classes, can delete non-significant classes, can
joint classes and can introduce his knowledge to the automatically generated
signal description by naming the classes, using established notions.
optimal application of the portable the physician uses a two or multi step
the first step, the recognition procedure runs on the stationary system
HeartScope with high performance and visual evaluation by a qualified expert, a
the second and further steps are used the portables PhysioCord for mobile
ECG-monitoring and recognition of possible risk situations.
methods for signal recognition worked with the highest accuracy, which employs
the latest technologies in the fields of Data Base (DBS) and Knowledge
Management Systems (KMS). Each incoming signal is stored in the DBS. These
signals are segmented and indexed by the time and by the segment number. The
classification is achieved using DB&KMS for the acquisition of personal
knowledge in direct communication between the user and the DB&KMS. These
stored and indexed signals in context with the introduced knowledge of the
expert build up the background knowledge for a new quality of signal recognition.
is the demonstrated strategy, that the physician is aided by the computer
system. The physician can introduce his own experience and evaluations using the
User Interface and the DB & KMS.
responsibility is in the hand of the physician, aided by powerful Information
and Communication Technologies.
progress of computer science has brought with it additional demands for the
integration of various technologies onto the portable computer and at the same
time has given rise to many novel applications. In the application presented
here, the minimized portable PhysioCord is used with a standardized
interface connecting the amplifier for ECG and other signals. Wireless data
transmission is used.
patterns which characterize the ECG and the risk situations are transmitted from
the high-performance HeartScope and are the basis for the next step: The
portable recognition system with low power technology is taught by templates for
marked details, ECG-substructures, heart beats and class descriptions of the
same patient with the same electrode positions. The recognition rate using this
strategy is asymptotic near 100%. The recognition procedure is implemented on
the high performance portable of the next generation, include high performance
computer power and the DB&KMS for signal recognition in real-time named
PhysioCord II. In this way, the system can recognize risk situations in
real-time with highest accuracy, while connecting the rescue organization by
handy-function and transmit relevant data for evaluation and decision making by
the physician. The receiver has a powerful Data Base & Knowledge Management
System with direct access. The physician can evaluate the received signals in
context with the course of the illness and the stored signals in the database.
He can decide with high accuracy in minimum time duration. The optimisation of
this decision process is the key process in this step. This creates a high
chance for saving risk patients in optimal time as illustrated in Fig 8.
further development requires highest performance of Information and
Communication Technologies, miniaturisation and lowest power consumption. The
aim is the integration of the power of the high performance stationary system in
the portable by using innovative technologies as
high precision analogue systems for data acquisition,
integrated multiprocessor systems in one chip,
FPGA with integrated processor core and
integrated broadband wireless systems.
All these high performance systems must
be designed with lowest power consumption for the portables.
next Generation System Architecture is characterised by high performance
portables. The portable is integrated by wireless data transmission technology
for the direct communication in risk situations with high data transmission
rates. The high performance stationary system with DB & KMS supports the
physician by the evaluation of the incoming signals in risk situations.
products, the BrainScope, the HeartScope in combination with the PhysioCord
are realised as Computer Aided Systems for patient monitoring, named Patient
Monitoring Systems - PatiMonS. This strategy is outlined in more detail:
(C) can be considered as the most critical aspect in such Computer Aided Systems
for different applications. Every component of the system relies on fast,
accurate and secure communication links. In order to better handle the
communications tasks, they have been divided into several meaningful entities:
C1: Communication physician – patient ;
C2: Communication between
Communication patient and computer technology;
physician and computer technology by the user interface;
Communication between physicians;
Communication between the PatiMonS’s;
acquisition and actions of the PatiMonS.
C4 has a special state and is used for the for
acquisition of the knowledge of the physicians and is important for the
definition of the responsibility, e.g. the physician must have all possibilities
for the evaluation of the state of the patient by using the actual incoming data
in context to the stored data and knowledge in the DB&KMS. The exchange of
data and knowledge between the physicians (C5) by direct or technical supported
communication (C5) is an important feature of the designed communication
System Architecture of CARiManS - Computer
Aided Risk Management System essentially handles all issues for the
distributed operation of Risk Management Systems. This operation is necessary as
to allow the Communication
network to be web based, scalable, platform independent, and easy to use.
involve the following components:
· UI: The User Interface considers the communication with the Risk Manager (C4), aiming the controlling and is able to transfer the expert’s knowledge to CARiManS and realise the communication of the Monitored System with the CARiManS (C3). It also enables the communication between the Risk Manager and the Monitored System (C1) as to enable control of the former to the operation of the latter.
MS: The unit Microsystems and
Sensors handles the acquisition of the Monitored System signals as well as the
controlling of the actuators e.g. multimedia technology for stimulation of
patients in psychophysiological experiments.
KM: Data Base & Knowledge Management System; the incoming
signals are stored in the DB, indexed by time, recognised events, and structures.
The KMS includes the knowledge of the experts, especially in context to the
stored signals. XML-technologies are used for the design of the KMS and for the
communication between technical systems.
PD: Portable Devices, signal
conditioning and conversion;
TI: Transmission and
Interfaces, security of data transmission; TI include terrestrial trucked radio or satellite technology for wide
area communication, wireless LAN adhoc
networks for disaster site hot spots, and personal or body area networks for
frontline personnel, allowing them to act as data sources and synchronise by
means of smart connected devices, e.g. robust mobile terminals and sensors.
SP: Signal Processing; modules
for signal analysis, classification, recognition, and evaluation. Special
modules are designed for real-time signal recognition.
CARiManS is designed for an integrated and improved risk management, providing
an infrastructure that allows for horizontal and vertical information flow from
the patient in risk up to the physician by means of a multi-level wireless
signal and data transmission and communication infrastructure, as well as
integrated applications that reflect the currently organizational structure
adequate to the rescue effort.
the concepts of direct communications among students, teacher, and computer,
CARiManS special configurations are used for teaching and knowledge
dissemination, for training and mentoring purposes, or simply for knowledge
transfer, accessible from every region, Europe and elsewhere.
are two scenarios:
the case that a
physician pushes his newly acquired knowledge towards his colleagues (teaching);
opposite case, where a teacher or a student asks for state-of-the-art knowledge
input from other experts (learning).
modular architecture, consistent, multidisciplinary, and robust design of its
components offers a significant impact on areas including skill-profiling of
every professional accessing the CATS (Computer-Aided Teaching System),
accessing the DB & KMSs (Data Base & Knowledge Management System) via
web based user friendly interfaces, automatic notification of important
knowledge as soon as new knowledge becomes available.
are important tools for spreading of knowledge and excellence.
F. Crick: The Astonishing Hypothesis. The Scientific Search for the Soul.
Charles Scribner's Sons New York 1993.
G. Roscher: Informationsprozesse in technischen Systemen - Modelle und Methoden
zur Erforschung des menschlichen Denkens. In Maas, J. F. Das sichtbare Denken,
Modelle und Modellhaftigkeit in der Philosophie und den Wissenschaften. Rodopi
1993, S. 153 - 179. http://www.ICSRoscher.de/maas.htm
G. Roscher, W. M. Herrmann, K. Henning, D. Wendt, S. Fechner, F. Godenschweger,
C. Weiß, E. Abel, A. Rijhwani, J. Martinez, N. Dahan, A. Karavas: Real-time
recognition of EEG-activity as basic for a new strategy of psychopysiological
experiments gives new insight in information processes in man. Clinical
applications of advanced EEG data processing. Rome, May 8-9 1995, p. 47. http://www.ICSRoscher.de/Rom.htm
G. Roscher: Mustererkennung in komplexen Signalen in Echtzeit durch die
Anwendung der Methoden der Künstlichen Intelligenz und den Einsatz leistungsfähiger
Multiprozessorsysteme. Forschungsbericht der DFMRS 1996, ISSN 0944-694X, p.
5. G. Roscher, G. Pogrzeba, D. Emde, F. Neubauer: Application of a Multi-Processor System for Recognition of EEG-Activities in Amplitude, Time and Space in Real-Time. In: E. H. D’Hollander, G. R. Joubert, F. J. Peters, U. Trottenberg (Eds.): Parallel Computing: Fundamentals, Applications and New Directions. 1998 Elsevier Science B. V. p. 89-96. http://www.ICSRoscher.de/Parco.htm
G. Roscher; A. Brattström; S. Cho;
F. Francesconi; H. Hinrikus; R. Mauser; M. Packianather; G. Pogrzeba; A.
Yankovskaya; V. Zvegintsev: Real-time Recognition of ECG by Using Powerful
Information and Communication Technology for Intelligent Monitoring of Risk
Patients. VDE-Verlag, Proceedings Volume 2, MICRO.tec 2000, Applications,
Trends, Visions, VDE World Micro-technologies Congress, September 25.-27. 2000,
EXPO 2000, Hannover, Germany, p. 759-762. http://www.ICSRoscher.de/EXPO2.htm
7. G. Roscher et al.: CAEPS - Computer Aided Environment Protection System. Expression of Interest, Sixth Framework Programme of the European Community for research, technological development and demonstration activities. June, 7. 2002. http://eoi.cordis.lu/docs/int_37066.pdf
G. Roscher et al.: PatiMon - Patient Monitoring. Expression of Interest, Sixth
Framework Programme of the European Community for research, technological
development and demonstration activities. June, 7. 2002. http://eoi.cordis.lu/docs/int_37068.pdf
G. Roscher et al.: RecoPhone - Recognition of Phonemes. Expression of Interest,
Sixth Framework Programme of the European Community for research, technological
development and demonstration activities. June, 7. 2002. http://eoi.cordis.lu/docs/int_37066.pdf
S1: Marcia Barinaga: „New Ion Channel May Yield Clues to Hearing. Science, 24. March 2000, Vol 287, p 2132-2133 and Science, 24. March 2000, Vol 287, p 2229-2234: „By studying the electrical currents passing through the membranes of hair cells as they are stimulated, they learned that hair-cell channels are stunningly fast, opening up within microseconds, compared to the milliseconds needed by biochemically activated channels. They are also exquisitely sensitive to the slightest movement and to direction; they open when the tip of the cell’s cilia bundle is deflected by a mere atom’s width – akin to bending the tip of the Eiffel Tower by the width of your thumb. If the cilia bundle moves one way, the channel opens; the other way and it shuts. The channels are also able to register tinny cilia movements on top of a larger constant deflection – a trait that lets us discern meaningful sounds from background noise.”