TUGraz OPEN Library

Kollmann, René

Neugebauer, Georg

Kretschmer, Florian

Truger, Barbara

Kindermann, Helene

Stoeglehner, Gernot

Ertl, Thomas

Narodoslawsky, Michael

Renewable energy from wastewater - Practical aspects of integrating a wastewater treatment plant into local energy supply concepts

Bolivar, Juan Manuel

Eisl, Ingrid

Nidetzky, Bernd

Advanced characterization of immobilized enzymes as heterogeneous biocatalysts

Krämer, Christina

Ungerböck, Birgit

Mayr, Torsten

Development of a Fully Integrated FallingFilm Microreactor for Gas–Liquid–SolidBiotransformation With Surface ImmobilizedO2-Dependent Enzyme

Ofner, Patrick

Schwarz, Andreas

Pereira, Joana

Müller-Putz, Gernot R.

Movements of the same upper limb can be classified from 
low-frequency time-domain EEG signals 
P. Ofner1, A. Schwarz1, J. Pereira1, G. R. Müller-Putz1*
1Institute of Neural Engineering, Graz University of Technology, Graz, Austria
*Stremayrgasse 16/IV, A-8010 Graz, Austria. E-mail: gernot.mueller@tugraz.at
Introduction: Brain-computer interfaces (BCIs) can be used to control neuroprostheses of spinal cord injured 
(SCI) persons. A neuroprosthesis can restore different movement functions (e.g., hand open/close, 
supination/pronation etc.), and requires a BCI with a sufficiently high number of classes. However, sensorimotor 
rhythm-based BCIs can often only provide less than 3 classes, and new types of BCIs need to be developed. 
Since a couple of years, a new EEG feature has evolved: low-frequency time-domain signals. For example 
movement trajectories [1] and movement directions [2] were decoded using this feature. In the present study, we 
investigated whether low-frequency time-domain signals can also be used to classify several (executed) 
hand/arm movements of the same limb. A BCI relying on the imagination of such movements may be used to 
control a neuroprosthesis more naturally and provide a higher number of classes. 
Material, Methods and Results: We recorded the electroencephalogram (EEG) using 61 channels and movement 
data from 15 healthy subjects using g.USBamps (g.tec, Austria) and ArmeoSpring (Hocoma, Switzerland). The 
subjects sat in a chair with the arm supported by the ArmeoSpring. We instructed subjects to execute hand 
open/close, supination/pronation, and elbow extension/flexion movements according to cues presented on a 
computer screen, i.e., 6 movement types/classes. In total, 360 trials (60 trials per class) were recorded. We 
removed independent components contaminated with artifacts and down-sampled data to 16 Hz. Subsequently, 
we referenced to a common average reference and applied a 0.3 – 3 Hz 4th order zero-phase Butterworth band-
pass filter to extract the low-frequency signals. Finally, we time-locked the data to the movement onset and 
classified the EEG with a shrinkage regularized linear discriminant analysis. To avoid overfitting we applied a 
10x10-fold cross-validation. Fig. 1 shows the classification accuracies of all subjects and the average. The 
average classification accuracy peaked at 0.0625s after the movement onset with a classification accuracy of 
37%. Classification accuracies were significant above 24% (p=0.05, Bonferroni corrected for multiple time-
points, Adjusted Wald interval). 
Figure 1. Classification accuracies of all 15 subjects. Time point 0s corresponds to the movement onset; thick black line is the average; 
horizontal solid line is the chance level; horizontal dashed line is the significance level.
Discussion: We have shown that low-frequency time domain signals can be used to discriminate between 
different movements of the same upper limb. Movement accuracies peak after the movement onset but reach 
significantly high classification accuracies before the movement onset. This shows that upcoming movements 
can be classified from the movement planning phase. This is crucial for a BCI applicable for end users with SCI 
who cannot execute all movements anymore. However, the classification accuracies are rather low and further 
studies have to investigate whether user training improves classification accuracies. 
Significance: Low-frequency time-domain signals contain information about upcoming movements and may 
serve as a control signals for future neuroprostheses. This would allow more degrees-of-freedom and a more 
natural control. 
Acknowledgements: This work is supported by the European ICT Programme Project H2020-643955 
“MoreGrasp” and the ERC Consolidator Grant “Feel Your Reach”. 
References 
[1] Bradberry T J, Gentili R J, Contreras-Vidal J L. Reconstructing Three-Dimensional Hand Movements from Noninvasive 
Electroencephalographic Signals. J Neurosci, 30(9): 3432-3437, 2010. 
[2] Lew E Y, Chavarriaga R, Silvoni S, Millan J del R. Single trial prediction of self-paced reaching directions from EEG signals. 
Front Neurosci, 8(222), 2014. 
DOI: 10.3217/978-3-85125-467-9-69Proceedings of the 6th International Brain-Computer Interface Meeting, organized by the BCI Society
Published by Verlag der TU Graz, Graz University of Technology, sponsored by g.tec medical engineering GmbH 69


Movements of the same upper limb can be classified from low-frequency time-domain EEG signals

Utts, Stanislav

Yamashita, Koji

Martinez , Sergio

Renner, Herwig

Carvalho Martins, Jose

Aristidou, Petros

Van Cutsem, Thierry

ПОДХОДЫ К МОДЕЛИРОВАНИЮ ГЕНЕРАТОРОВ С ИСПОЛЬЗОВАНИЕМ ПРЕОБРАЗОВАТЕЛЬНОЙ ТЕХНИКИ

Weinberger, Martin

Tomasch, Ernst

Hoschopf, Heinz

Steffan, Hermann

Spitzer, Peter

Kleewein, Franz

Causation of traffic accidents with children from the perspective of all involved participants

Kieseberg, Peter

Weippl, Edgar

Holzinger, Andreas

ERCIM NEWS
www.ercim.eu
Number 104 January 2016
Special theme
Tackling Big Data 
in the 
Life Sciences
Also in this issue:
Research and Society: 
“Women in ICT Research and Education”
more than 60% of these to be true ADR
tweets.
Future improvements
Automatic detection of ADRs on
Twitter (or other social media channels)
is still a very young discipline, which
only started some five years ago. There
are only a few teams working on the
topic at the moment, and a first large-
scale benchmark dataset was only pub-
lished in 2014 [3]. However, we expect
a significant improvement in detection
rates in the near future, owing in part to
the existence of several new technolo-
gies in machine learning, such as word
embedding and deep learning. These
have already been successfully applied
to other text analysis tasks and have
improved existing benchmark scores
there. Applying these technologies to
ADR detection will probably help to
increase the detection rate significantly.
In addition, our team is working on a
system that not only analyzes the text of
a tweet, but also its context: the timeline
of the user, other messages in the same
geographic or temporal context etc.
This will allow us to “step back” from
an isolated event (a single tweet) and
see the “whole picture” of the discourse
on Twitter. 
References:
[1] J. Lazarou, B.H. Pomeranz, and
P.N. Corey: “Incidence of Adverse
Drug Reactions in Hospitalized
Patients: A Meta-analysis of
Prospective Studies”, JAMA
279(15):1200-1205, 1998. 
[2] Pacific Symposium on Biocomput-
ing: http://psb.stanford.edu/work-
shop/wkshp-smm/
[3] R. Ginn et al.: “Mining Twitter for
Adverse Drug Reaction Mentions:
A Corpus and Classification
Benchmark”, BioTxtM, 2014.
Please contact:
Mark Cieliebak
School of Engineering
Zurich University of Applied Sciences
(ZHAW) 
Tel: +41 58 934 72 39
E-mail: ciel@zhaw.ch
ERCIM NEWS 104   January 201632
Special Theme: Tackling Big Data in the Life Sciences  
Figure 1: Typical system for ADR detection using machine learning.
The “doctor in the loop” is a new para-
digm in information driven medicine,
picturing the doctor as authority inside a
loop with an expert system in order to
support the (automated) decision
making with expert knowledge. This
information not only includes support in
pattern finding and supplying external
knowledge, but the inclusion of data on
actual patients, as well as treatment
results and possible additional (side-)
effects that relate to previous decisions
of this semi-automated system.
The concept of the "doctor in the loop"
is basically an extension of the increas-
ingly frequent use of knowledge dis-
covery for the enhancement of medical
treatments together with the “human in
the loop” concept (see [1], for instance):
The expert knowledge of the doctor is
incorporated into "intelligent" systems
(e.g., using interactive machine
learning) and enriched with additional
Trust for the “Doctor in the Loop”
by Peter Kieseberg, Edgar Weippl and Andreas Holzinger
The "doctor in the loop" is a new paradigm in information driven medicine, picturing the doctor as
authority inside a loop supplying an expert system with data and information. Before this paradigm
is implemented in real environments, the trustworthiness of the system must be assured.
ERCIM NEWS 104   January 2016 33
information and expert know-how.
Using machine learning algorithms,
medical knowledge and optimal treat-
ments are identified. This knowledge is
then fed back to the doctor to assist
him/her (see Figure 1).
Manipulation Security and Trust 
The implementation of the doctor in the
loop concept has met several challenges
– both of a technical nature and in other
areas. One challenge is gaining the
acceptance of such systems by doctors
themselves, who are often not
researchers, but medical practitioners.
While privacy and security in biomed-
ical data applications have been dis-
cussed extensively in recent years (see
[2] for an overview), the topic of trust
has been neglected. Nevertheless, it is
very important that the trustworthiness
of the systems can be guaranteed in
order to make the abstract concept of a
doctor in the loop practicable.
In this medical context, it is absolutely
crucial to ensure that the information
sent by the doctor to the machine
learning algorithms cannot be manipu-
lated following its submission. In order
to guarantee this, a new approach for
protecting the underlying data based on
a hash chain has been proposed explic-
itly for doctor in the loop systems (see
[3]). This approach takes advantage of
the fact that large amounts of data are
typically stored in databases. While
these are often identified with the
abstract data model, they are in reality
complex software systems providing a
multitude of internal mechanisms that
have various functions, such as
enhancing performance. This approach
utilizes the transaction mechanism for
providing transaction safety (i.e., if a
crash occurs, the database is brought
back into a well-defined state) in order
to protect the information sent to the
system by the doctor against manipula-
tion, even if the manipulation is carried
out by the administrator of the database. 
Legal Issues
Legal issues are often overlooked by
purely technical solutions, but are vital
for any doctor that is actually partici-
pating in such an approach as expert.
Important questions that need to be
addressed include: What information
can be shared between the doctor and
the expert system, and what levels of
data and privacy protection need to be
applied? And who is responsible for the
correctness of the results derived from
the combination of human knowledge
and machine learning algorithms?
Although this is an important field, no
guidelines are currently available. The
issue is further complicated by the dif-
ferences between national legislations
even between member states of the
European Union. Defining workflows
that clinical doctors can reliably apply
without the fear of prosecution lies thus
in the focus of the planned RDA
(Research Data Alliance) Workgroup
“Security and Trust” that held its “Birds
of Feather” session on the 6th RDA
Plenary on September 24th in Paris
[L1]. One of the major goals of this
workgroup is to draw together a set of
best practices and guidelines in the area
of data driven medical research.
Special focus in CBmed
CBmed [L2] is an interdisciplinary
research centre, founded in 2014, with
the aim of providing biomarker
research, mainly in the areas cancer,
metabolism and inflammation. One of
the core features of this centre is the
tight incorporation of ICT as a hori-
zontal research area that provides data
and techniques to all other projects.
This does not only apply to applications
associated with the doctor in the loop,
but also other areas, such as data and
privacy protection, the development of
new data mining techniques and tools
for the efficient analysis of large
amount of “–omics” data.
For the CBmed consortium, the model
of the doctor in the loop offers abundant
possibilities, especially in the area of
data driven research. Considering the
recent surge in big data related research
and developed tools, this approach is
expected to be one of the major drivers
in medical research in the years to
come. 
Links:
[L1]: https://rd-alliance.org/rda-work-
ing-group-data-security-and-trust-
wgdst-p6-bof-session.html 
[L2]: http://www.cbmed.org/en/ 
References:
[1] W. S. Levine, ed.: “The control
handbook”, CRC press, 1996.
[2] P. Kieseberg, H. Hobel, S.
Schrittwieser, E. Weippl, A.
Holzinger, “Protecting anonymity in
data-driven biomedical science”, in
“Interactive Knowledge Discovery
and Data Mining in Biomedical
Informatics” (pp. 301-316),
Springer Berlin Heidelberg, 2014.
[3] P. Kieseberg, J. Schantl,
P.Fruehwirt and E. R. Weippl, A.
Holzinger: “Witnesses for the
Doctor in the Loop," in 2015
International Conference on Brain
Informatics & Health (BIH), 2015.
Please contact: 
Peter Kieseberg
SBA Research, Vienna, Austria
E-mail: pkieseberg@sba-research.org
Figure 1: The Doctor in the Loop.


Trust for the Doctor-in-the-Loop

Cloete, Jan Hendrik

Cloete, Schalk

Municchi, Federico

Radl, Stefan

Amini, Shahriar

The sensitivity of filtered Two Fluid Models to the underlying resolved simulation setup

Eisert, Jens

Eisler, Viktor

Zimborás, Z.

Entanglement negativity bounds for fermionic Gaussian states

Plass, Markus

Kickmeier-Rust

Interactive Machine Learning (iML): a challenge for
Game-based approaches
Andreas Holzinger, Markus Plass, Michael D. Kickmeier-Rust
Holzinger Group HCI-KDD, Institute for Medical Informatics, Statistics and Documentation
Medical University Graz, Austria
Knowledge Technologies Institute, Graz University of Technology, Austria
a.holzinger@hci.kdd.org
2-page Extended Abstract CiML-2016-Workshop, Barcelona, Spain
Figure 1: iML can be defined as algorithms that interact with agents and can optimize their learning
behavior through these interactions, where the agents can also be humans [1]. These humans-in-the-
loop can be beneficial in solving computationally hard problems
The goal of the ML-community is to design and develop algorithms which can learn from data and
improve with experience over time. However, the application of such automatic machine learning
(aML) approaches in the complex biomedical domain seems elusive in the near future, and a good
example are Gaussian processes, where aML (e.g. standard kernel machines) struggle on function
extrapolation problems - which are trivial for human learners [2]. Psychological research indicates
that human intuition is not an ’esotheric’ concept; much more, it is based on distinct behavioral
and cognitive strategies that developed evolutionary over millions of years [3]. For improving ML,
we need to identify the concrete mechanisms and we argue that this can be done best by observing
crowd behaviors and decisions in gamified situations.
We have proved experimentally that the iML approach can be used to advance current Traveling
Salesman Problem (TSP) solving methods [4]. TSP is important, as it appears in a number of prac-
tical problems in biomedical informatics, e.g. the native folded three-dimensional conformation of
a protein is its lowest free energy state and both a two- and three-dimensional folding processes as
a free energy minimization problem belong to a large set of computational problems, assumed to
be very hard (conditionally intractable) [5]. For our experiment we applied the Ant Colony Opti-
mization (ACO) - which is a fully automatic (aML) approach without any human intervention: The
ants walk around and update the global weights after each iteration. This procedure is repeated a
distinct number of times. Now, following the iML-approach the human can open the black-box and
1
manipulate the algorithm by changing the behavior of the ants; based on the Inner Ant System for
TSP [6] this could be understood as adding or removing of pheromones on a trail between nodes of a
graph. Consequently, the trails become more or less interesting for the ants and there is a high prob-
ability that they will consider trail changes. Due to the fact that we have previous experience with
gamification for education [7] we are currently gamifying our experiment: To achieve an intuitive
interaction the problem is presented as a game, where the user has to reduce the length of trail, by
changing the amount of pheromones on the lines between the nodes. The implementation is based
on Java-Script, with the advantage of being a browser based solution with the benefit of platform
independence and no installation requirements. For the test-setup we used 30 ants, 250 iterations.
For the other parameters including evaporation we choose fixed default values, this makes it easier to
compare the results. Humans performance is excellent for problems with a size of about 100 nodes
these could be solved by humans intuitively, for problems with x-thousands of nodes it is impossible
to get an overview and to find useful solutions within acceptable time. The idea is to split a large
2D-problem into smaller overlapping sub-problems which can be solved separately, and through the
overlap they can be merged after the interaction together, this approach will allow us to work also
with large data sets from the biomedical domain (e.g. [8]).
The challenge is in 2D-visualization, i.e. to represent the data, particularly larger and complex data
sets, in a form which is still comprehensible by a human. For this purpose we are administering
a software engineering student challenge within our university course at our faculty of computer
science; but we could also extent this challenge to international level. By representing the data in
form of easy to understand 2D graphics, humans may find possible solutions quickly and intuitively.
By this means, distinct decision and problem solving patterns can be identified, which so far have
been considered being this vague concept of human intuition, hence can contribute to the outcomes
of the aML-Algorithm in an interactive way. By this approach we believe to help to answer the
difficult question of ”What is interesting?” to find a better solution than the aML-approach would
find in the same time, given that this time span is reasonably limited. Our first analyses showed, that
on two of the three test data sets the iML approach gives better results than the aML approach, this
results are reached by only 2.4 (in mean) human interactions.
References
[1] A Holzinger. Interactive machine learning for health informatics: When do we need the human-
in-the-loop? Brain Informatics, 3(2):119–131, 2016. doi: 10.1007/s40708-016-0042-6.
[2] A G Wilson, C Dann, C Lucas, and E P Xing. The human kernel. In Corinna Cortes, N.D.
Lawrence, D.D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information
Processing Systems, NIPS 2015, volume 28, pages 2836–2844, 2015.
[3] M D Lieberman. Intuition: a social cognitive neuroscience approach. Psychological Bulletin,
126(1):109–137, 2000. doi: 10.1037/0033-2909.126.1.109.
[4] A Holzinger, M Plass, K Holzinger, GC Crisan, CM Pintea, and V Palade. Towards inter-
active machine learning (iml): Applying ant colony algorithms to solve the traveling sales-
man problem with the human-in-the-loop approach. In Springer Lecture Notes in Com-
puter Science LNCS 9817, pages 81–95. Springer, Heidelberg, Berlin, New York, 2016. doi:
10.1007/978-3-319-45507-56.
[5] P Crescenzi, D Goldman, C Papadimitriou, A Piccolboni, and M Yannakakis. On the com-
plexity of protein folding. Journal of computational biology, 5(3):423–465, 1998. doi:
10.1016/S0092-8240(05)80170-3.
[6] Camelia Pintea, D Dumitrescu, and PC Pop. Combining heuristics and modifying local infor-
mation to guide ant-based search. Carpathian Journal of Mathematics, 24(1):94–103, 2008.
[7] M Ebner and A Holzinger. Successful implementation of user-centered game based learning
in higher education: An example from civil engineering. Computers and Education, 49(3):
873–890, 2007. doi: 10.1016/j.compedu.2005.11.026.
[8] I Guyon, D Battaglia, A Guyon, V Lemaire, J G Orlandi, B Ray, M Saeed, J Soriano, A Stat-
nikov, and O Stetter. Design of the first neuronal connectomics challenge: From imaging to
connectivity. In 2014 International Joint Conference on Neural Networks (IJCNN), pages 2600–
2607. IEEE, 2014.
2


Interactive Machine Learning (iML): a challenge for Game-based approaches

Graz University of Technology

Not a member yet