1. Evaluation of Prediction Models in Medicine
  2. The Clinician-Data Scientist Dyad: Understanding Both for an Exceptional Convolution
  3. Medical Information Retrieval
  4. Argumentation Technology in Medicine


  1. Knowledge Representation for Health Care / Process Process-oriented Information Systems in Health Care (KR4HC/ProHealth) 2019
  2. Modern Technologies and Artificial Intelligence in Orthopedics and Rehabilitation
  3. Transparent, Explainable and Affective AI in Medical Systems (TEAAM)

Tutorial 1: Evaluation of Prediction Models in Medicine

Ameen Abu-Hanna, University of Amsterdam, The Netherlands


The reliable prediction of outcomes from disease and treatment is becoming increasingly important in the delivery and organisation of health care. Advances in personalised medicine are catalysing the search for new prognostic markers. The uses of outcome predictions range from the level of individual patients, where they help doctors and patients to make treatment choices, to patient populations, where they support health-care managers in planning and allocating resources.

This tutorial focuses on methodologies for quantitative assessment of the performance of prediction models. The key to quantitative evaluation is the use of reliable methods for obtaining valid performance measures of unseen data with well-defined characteristics.

The tutorial will clarify the relevant methods and the relationship between them using conceptual and mathematical frameworks. It is explained under which circumstances specific methods are applicable and when they are not. In addition, attention will be paid to the various choices in the design of model evaluation procedures, and the relationship between model evaluation and the purposes for which a model has been built. All methods are illustrated with real-world examples from the domains such as cardiac surgery and intensive care medicine.

Tutorial 2: The Clinician-Data Scientist Dyad: Understanding Both for an Exceptional Convolution

Anthony Chang, Children’s Hospital of Orange County, USA

Despite the surge of interest in artificial intelligence in healthcare and medicine, there continues to be a sizable chasm between the clinicians and the data scientists. There is a fundamental lack of understanding of each other’s culture and domain. In addition, there is insufficient appreciation of each other’s knowledge and expertise. This multi-dimensional misalignment results in a myriad of missed opportunities for significant and meaningful applications of data science and artificial intelligence in healthcare and medicine. With synergy and convolution of clinical medicine and data science, the future of biomedicine will be transformed into intelligence-based medicine.

Tutorial 3: Medical Information Retrieval

Lynda Tamine, University of Toulouse UPS-IRIT, France
Lorraine Goeuriot, University of Grenoble Alpes, France

The rapid increase of medical information sources and volumes (eg.,electronic health records, medical forums, scientific literature) on the one hand, and the diversity of users and tasks (eg., diagnosis, clinical trial, health care, etc.) on the other hand, has renewed the need for a future generation of medical information retrieval (IR) systems with the objective of supporting clinical decision processes that provide adequate support for both novices (eg., patients and their next-of-kins) and experts (eg., physicians, clinicians) tackling complex tasks. This trend gives rise to relevant opportunities for the medical community but opens crucial challenges for health professionnals as well as patients and their families. During the tutorial, we will first present an overview of IR fundamentals and then report the major findings which highlight the potential of medical and health-related search traces to better understand the users, their information needs and their use of search results for daily health care. Afterwards, we discuss representative state-of-the-art techniques and methods which support medical IR systems; in addition to seminal works, we will present some recently proposed concept representation techniques which rely on research advancements in deep learning. We will also address the challenge of evaluating medical IR tasks and present major campaigns, resources and lessons learned. Finally, we will conclude the tutorial by a roadmap and a discussion phase.

Tutorial 4: Argumentation Technology in Medicine

Philipp Cimiano, Bielefeld University, Germany
Laura Moss, University of Glasgow, UK
Olivia Sanchez-Graillet, Bielefeld University, Germany
Basil Ell, Bielefeld University, Germany

In medicine, complex decisions are made by clinicians often in uncertain conditions. The field of argumentation provides a formal framework for modeling human collaborative deliberations, interchanging arguments in favor or against some conclusion based on incomplete or inconsistent information. Argumentation theory has become an important research field in Artificial Intelligence (AI). The relationship between computer science and the area of philosophy focused on arguments has led to the emergence of a new interdisciplinary field called computational dialectics, argumentation technology, or argument-based computing.

Argumentation has been investigated as a tool for providing clinical decision support, changing health-related behaviors, tailoring explanations, advising patients on treatment regimes, as well as for designing agents working in cooperation within healthcare teams.

In this tutorial, we provide an introduction into argumentation technology for medicine as well as an overview of the main techniques, use cases and applications thereof. We focus in particular on two main applications. First, we discuss how argumentation technology can be used as a basis to generate hypotheses and explanations for anomalous patient responses. Second, we discuss how arguments can be used to support evidence-based decision-making by using arguments as a tool to aggregate evidence across multiple clinical studies.

Workshop 1: Knowledge Representation for Health Care / Process Process-oriented Information Systems in Health Care (KR4HC/ProHealth) 2019

Mor Peleg, University of Haifa, Israel
Mar Marcos, Universitat Jaume I, Spain
Richard Lenz, University of Erlangen and Nuremberg, Germany


In the last years we have witnessed the increasing incorporation of computer technologies for knowledge representation and process modeling to improve health care and to provide high-quality modern clinical services.

These technologies remain at the very core of other medical informatics areas such as decision support systems, e-health, m-health, smart health, simulation, clinical alarm systems, electronic health care records, patient-centered care, modeling, standardization, and quality assessment.
The Joint International Workshop KR4HC-ProHealth in 2019 is the seventh time that two separate research communities merge to address common medical issues, to discuss about new trends, and to propose solutions to health care issues by means of the integration of knowledge representation and process management technologies as a contribution of the advance of medical informatics.

As part of medical informatics, the knowledge-representation for health care (KR4HC) view focuses on representing and reasoning with medical knowledge in computers to support knowledge management, clinical decision-making, health care modeling and simulation. This community aims at developing efficient representations, technologies, and tools for integrating the important elements that health care providers work with: Electronic Medical Records and healthcare information systems, clinical practice guidelines, and medical vocabularies.

As part of business process management, the process-oriented information systems in healthcare (ProHealth) view focuses on using business process management technology to provide effective solutions for the management of healthcare processes. This community aims at adapting successful process management solutions to health care processes and needs, with a particular interest in organization, optimization, cooperation, risk analysis, flexibility, re-utilization, and integration of health care tasks and teams.

Workshop 2: Modern Technologies and Artificial Intelligence in Orthopedics and Rehabilitation

Marcin Domżalski, Medical University of Lodz, Polish Arthroscopic Society, Poland
Wojciech Glinkowski, Medical University of Warsaw, Polish Telemedicine and eHealth Society, Poland
Przemysław Lisiński, Poznan University of Medical Science, Poland
Beata Mierzejewska, AISENS, Poland


Many health care systems in the world are facing challenges due to fast-growing aging population where residents of industrial countries are now living longer, but with multiple, often complex health conditions. Also due to the growing population, the orthopedic or rehabilitation specialist time availability per one patient is limited what’s making it often impossible to provide the patient with good, quality care with any delay, complex diagnoses or personalized treatment. A big part of the solution may lie in how and to what extent we take advantage of recent technological advances in medical informatics technology and related fields.

In Poland, for example, a patient who wants to schedule an appointment with an orthopedic specialist, orthopedic procedure or rehabilitation (all provided with the public healthcare system) has to wait a very long time – from months to years. This is a major problem not only for patients but also for orthopedic and rehabilitation specialists who due to this problem have to often work with their patients longer than necessary or even re-operate them. Telemedicine available also in rehabilitation could help to solve this problem today and in the future.
Another important matter in orthopedics and rehabilitation is the possibility of improving the quality of service due to new technologies, such as precise diagnostics, personalized long term treatment planning, precise surgery planning, custom made implants, AI-augmented electronic patient records for both orthopedic and rehabilitation specialists, etc.

Improving the quality of service by implementation of new technologies like Telediagnostics, teleconsulting, telemonitoring, VR, AR custom made implants, 3D, smart database, etc. could not only effect positively patients’ lives but also could save hospitals a lot of money.

The 1st Workshop on Modern Technologies in Orthopedics and Rehabilitation is a great opportunity for clinicians, researchers, product innovators and manufacturers to share their ideas and exchange their experience. During workshop, all the participants will also have the opportunity to cooperate together and learn how important for the future is their participation in new technological projects

 Workshop 3: Transparent, Explainable and Affective AI in Medical Systems (TEAAM)

Grzegorz J. Nalepa, AGH University of Science and Technology, Poland
Gregor Stiglic, University of Maribor, Slovenia
Sławomir Nowaczyk, Halmstad University, Sweden
Jose M. Juarez, University of Murcia, Spain
Jerzy Stefanowski, Poznan University of Technology, Poland


Medical systems highlight important requirements and challenges for the AI solutions. In particular, demands for interpretability of models and knowledge representations are much higher than in other domains. The current health-related AI applications rarely provide integrated yet transparent and humanized solutions. However, from both patient's and doctor's perspective, there is need for approaches that are comprehensive, credible and trusted. By explaining the reasoning behind recommendations, the medical AI systems support users to accept or reject their predictions. Furthermore, healthcare is particularly challenging due to medicine and ethical requirements, laws and regulations and the real caution taken by physicians while treating the patients. Improving individual's health is a complex process, requiring understanding and collaboration between the doctor and the patient. Building up this collaboration not only requires individualized personalization, but also a proper adaptation to the gradual changes of patient’s condition, including their emotional state. Recently, AI solutions have been playing an important mediating role in understanding how both medical and personal factors interact with respect to diagnosis and treatment adherence. As the number of such applications is expected to rapidly grow in next years, their humanized aspect will play a critical role in their adoption. This workshop will bring together researchers from academia and industry to discuss current topics of interest in interpretability, explainability and affect related to AI-based systems present in different healthcare domains.