1. Executive Summary

In the good old days knights bared women on their arms, and honor and dignity were prized above richness. In those days, every family knew the only man, who always relieved them in their pain, and came to help in every health problem, be that injury or delivery. This man knew the grandparents, he also knew the children. He was with the family in the hardest days of their life - he was the family doctor.

Many years have passed, and the profession of a family doctor is almost lost. There is no doctor who advises us from the birth till the very end, there is no specialist who knows every scratch on our body.

Nowadays, technology enriches our lives with new abilities - it helps us in work, in communication and, of course, it enables us to live healthier lives.

Imagine a world where you can consult with your private doctor using a computer on a daily basis, and where the power of many professionals from different parts of the world is concentrated in one place, giving the personal health assistant an unprecedented medical qualification, which raises from the unleashed power of the community of medical experts. In the distributed world of today, tools can give experts the ability to diagnose a patient together without meeting each other, at any convenient time or place. Let us assume for a moment that we have such a technology, which can help us to revive a profession of family doctor at the new level, where extremely powerful digital assistant plays the role or personal advisor, record keeper, and life saver - helping us live healthier and safer lives.

We present the Olive system - technology designed to make distributed medical consultations reliable and accessible - as simple as talking to a family doctor. We will discuss both social impacts of Olive and technical implementation.

2. Introduction

When we feel sick, we need to get the appropriate treatment - and it is essential that we get the right diagnosis before the treatment is started. While wise people usually consult a doctor, who helps determining a cause of the disease and advises the best treatment, many people intend to find alternative ways, like asking their friends if they had similar symptoms, or searching on the Internet. It is very important that people use the right tools to get diagnosed, which use Internet connectivity to spread the word of medicine, and not gossips.

Computers, mobiles phones and Internet connectivity are playing an increasingly valuable role in helping doctors and patients - and one of the ways is performing remote medical consultations. Virtual consultations approach, also known as telemedicine, is found to be effective in many cases like:

However, modern telemedicine technologies have a number of disadvantages:

By drawing the technologies from the area of Artificial Intelligence, we believe we can make things better.

3. Olive Project

3.1. Our research

A goal of our research was to develop a product able to make telemedicine consultations safe, useful and really effective. We wanted to give everybody an opportunity to get virtual consultations with professionals on a daily basis, without the need to visit a hospital. We wanted to exploit the power of expert community, to allow combining the expertise of individual professionals to achieve synergetic effect.

Our initial research concerned both scientific and technical areas. Main questions of interest were:

For example, suppose a person feels ill. Let us see the common procedure he/she will undertake to find the best treatment:

Our product, Olive, is based on the outcome of our research activities. First we will discuss the main idea behind the system, and later concentrate on the technical implementation.

3.2. The Power of Social Groups

We believe that the Olive System opens a brand new way of remote medical consultations. The system is based on multi-agent approach, where each expert in the Olive network is represented by his own personal agent called Dashboard (DB). DB-agents provide a formalized communications between experts, directed to solve a problem of making a diagnosis.

Specialists, through their personal assistant DB-agents form a network of experts over the Internet. All of them are specialists in their own field of medicine, and every consultation collects several professionals together to make the diagnosis more accurate.

To effectively allow a group of people work on solving one problem, we need to perform rather complex distributed reasoning, which is somehow directed towards the common goal. We use the blackboard approach, where all experts have access to some formalized ontological storage of patient's data, and steps in making the diagnosis are viewed as state changes in the blackboard data. This notion leads us to more formalized human-computer reasoning model described later in this document.

It was suggested by some philosophers and futurologists (among them V.F.Turchin, F.Heylighen, et.al.) that effective communications inside the group of professionals would be responsible for the formation of the "global brain" of the humankind. While the "global brain" is more than just distributed reasoning, we believe that Olive project does essentially contribute to the formation of emergent meta-intelligence of an expert group.

Another question to consider is how we can decide which expert we need to invite to a group. Most probably we compare the patient's symptoms and the knowledge on his diagnosis we have at each point with the profiles of specialists on the Olive network, calculating how relevant the symptoms are to that specific person's field of expertise. More formal definition of the value of similarity between a symptoms list and specialist's profile is given by the concept of ontological relevancy, and it implies that person's profile is annotated using ontology-based approach.

3.3. Ontology-Based Annotation

Let us closer examine the way experts' profile is described. Our model implies comparing the semantics of patient symptoms and with doctor's profile. This is done by using the notion of ontological relevancy, which is based on the mathematical model of description logics and its implementation in descriptive languages like RDF and OWL, which belong to the family of Semantic Web standards.

The profile of every specialist is constructed on the basis of some existing ontology. For instance if someone is willing to share that he is the cardiologist and his name is John Smith, the definition would be something like this: PROFILE is-subclass-of DOCTOR, PROFILE has-profession CARDIOLOGIST, PROFILE has-name "John Smith". 'CARDIOLOGIST' and 'has-profession' in this example are the names of the ontology concept and ontology role respectively. The string entity "John Smith" is used as an atomic primitive and does not take part in relevancy calculation.

There are a lot of developed ontologies for different aspects of human activities today. We have developed base Olive ontology by deriving it from the International Statistical Classification of Diseases and Related Health Problems (ICD-10), released and supported by the World Health Organization (WHO), and OpenGALEN, another industry-proven medical ontology.

We have also developed an ontological model of patient's health state, which describes the static snapshot of person's medical record, pretty much like a personal digital medical card, containing both initial symptoms and derived knowledge on the diagnosis. However, since it is ontologically formalized, it is possible to perform some automated reasoning and relevancy calculations on the patient's current health state record as well, and to use parts of the record as a basis for blackboard in distributed consultation.

Formalized health state could be described as a set of bindings to Olive ontology. In addition to formal bindings, local health state could also contain data which is necessary for consultation, but is not describable in the terms of ontology - like X-ray scans, or photographs.

3.4. Ontological Relevancy

When the patient or virtual consilium want to find some expert to consult with, Olive uses part of the current formalized health state (called a query) to find the right doctor. During the search, the ontological relevancy of symptoms is computed against doctor's profile. This is done by traversing the concept graphs in profile and query. Every concept used in profile definition is compared with every concept in the query with respect to the roles that were used to bind these concepts to profile or symptom. The distance in the graph to the most concrete common parent of every concept pair is used to compute this pair's relevancy as:

where - individual terms in profile description, - query terms, mcs - a most concrete common parent of two terms (which is always defined for each two terms, because terms form a complete lattice in terms of subsumption relation), r is a distance function that defines the number of direct subsumption steps between two terms in a chain, is a weight function that correlates with the roles of p and q in the profile and query, a, b and c are the coefficients that can be defined using the statistical analysis. In other words, total relevancy between profile and a query is computed as a weighted sum of the individual term relevancies, where weights are different for every role and are defined in the process of model testing and simulation.

3.5. Rule-based Reasoning

The notion of Ontological Relevancy is not always enough to determine the best professional according to the given health state. For example, we might want to take into account some common medical practices, like always consulting with an oncologist if a person has the urological disease and is older than 80 years. Thus, ontological relevancy is used in conjunction with rule-based reasoning model, where rules are described using RuleML Lite and are used during the stages of reasoning which mostly involve human experience. Rule model includes traversing the ontology for determining the subsumption of the concepts which are used in the left parts of the rules. The result of rule application is usually a set of suitable concepts that is added to formalized health state; therefore it is reasonable to perform rule-based inference for initial symptoms refinement before performing Ontology Relevancy computation.

3.6. Patients Routing

When the person wants to perform virtual consultation using Olive system, he typically performs the following standard steps:

As you can see, Olive uses the experience of a group of experts as well as computer's one, in terms of collaborative filtering, ontology relevancy and rule-based reasoning - we this process a human-computer reasoning.

3.7. Human-Computer Reasoning

The main idea of human-computer reasoning is to use an experience of self-organized social group of experts as well as accumulated one, using a collaborative filtering method.

Although the majority of the consultation problems suppose to be solved by actively using a human experience, it is possible to combine the experts reasoning with the conclusions of and expert system. Olive is a complex distributed reasoning system that utilizes the advantages of both automated and natural reasoning.

The process of distributed reasoning is based around a centralized ontological blackboard, which represents the environment for several assistant agents taking part in problem solving. The role of the blackboard is to make agents assemble their knowledge to build the solution step by step - i.e., the blackboard serves as a problem state, while agents specify the directions of search for the solution in the global state space graph.

Automated approach with formalized ontological blackboard allows Olive to accumulate the knowledge of professionals, and by using the methods of collaborative filtering show them the most frequent or similar solution taken by expert groups in the past. Experts that take part in the distributed reasoning process can use their own experience and the statistics of precedents which is provided for them by the system. Thus, in addition to human reasoning, we provide tips based on collaborative filtering and case-based reasoning approaches, as well as some rule-based and ontology-based reasoning.

Human-computer approach enables us to increase the synergistic effect of social group during the consultation, by integrating expert's knowledge not only along the space dimension (brining different experts together), but also along the time line (taking past experience into account).

3.8. Economical Model

In order to force patients and doctors to use the system, we need to provide some incentives for them. One of the ways to do that is to introduce some economical bonus model behind the system that will simulate the economical relationships in the real world to keep this artificial reasoning universe alive. The main principles of suggested economical model are the following:

We realize that the exact bonus policy of such a system needs more work - in particular, we need to ensure that money flowing from the patient's are enough to compensate for the bonuses of experts, and that some money stays inside the system to support its operation. This can probably be done in terms of taxes that expert pay from their profits. However, we believe that the proposed economical model can serve as a basis for future development, and can prove the concept that such virtual consultation universe is viable.

3.9. Confidentiality Policy

To provide some confidentiality of patients' diagnoses, treatment procedure and other private information, we design the system in such a way that formalized health state (also known as digital medical card) is stored on patient's local machine only, and any access to this information is done with the permissions from the owner. In addition, when consultation is performed, or information is stored in the Olive knowledgebase for collaborative filtering, it is effectively impersonated - experts do not need to know the name of their patient to perform accurate and unbiased diagnosis.

4. Inside Olive

4.1. Overall Architecture

Olive system is based on multi-agent approach. Every node in the Olive network - both for doctors and patients - is represented by a Personal Assistant Agent.

The following diagram provides an overview of the system architecture:


Fig. 1. Olive Physical Diagram

One physical patient is represented by one or two agents: Desktop and Mobile Personal Agents. Mobile system is intended to accumulate the health status information during the day and to maintain a treatment monitoring, instituted by a doctor.

A functionality of the system installed on the patient's desktop computer is extended with the ability to take part in remote consultation - a telecommunication session with doctors. Desktop Agent also collects the information from PDA Mobile assistant and possibly from some medical devices attached to the system that are able to perform simple tests (monitoring blood pressure, glucose level, pulse and some other medical information).

Experts are represented in the Olive network by Dashboard Agents. The main functionality of DB-agent is to provide a formalized communication between experts to solve the diagnosing problem and to effectively use the cumulative experience.

The idea of a blackboard system is that all data relevant to a given diagnostic session is put up on a formalized "blackboard" whose contents are visible to the set of consulting experts. Thus, the blackboard would initially consist of the set of symptoms obtained from the patient. Then, a set of agents (which represents doctors) look at the blackboard and each one decides if it has something to contribute to solving the problem, and if it does, it makes the appropriate change to the blackboard, leading to a new health state of the patient.

In the process of a consultation, the blackboard will contain more specific symptoms, then the diagnosis, and finally the treatment.

Adopting multi-agent approach with formalized blackboard increases scalability and availability of the entire system, because agents run on different computers, and they access the blackboard located on the patient's computer - thus there is no centralized location on the internet that experiences the significant load in the process of consultation, and acts as a bottleneck to the system.

The agents of Olive system communicate by exchanging formalized packets with each other and with the blackboard system. All supporting information is collected from and by so-called HealthWare, which is a central Olive server located somewhere on the internet. HealthWare does not take an active part in the consultation, but it contains the list of profiles of system users, and is responsible for collecting results for collaborative filtering and case-based reasoning.

4.2. Health Ware

All centralized functionality for user communication, doctor's search, etc. is provided by HealthWare server.

Health Ware provides:

Mostly of the HealthWare functionality is represented by web services, accessible through the Internet.

4.3. Collective Reasoning

On the picture below you can see a typical process of using blackboard-based collective reasoning:

Let us consider the diagnosing process:

  • Patient shares the symptoms on the blackboard
  • Spec-physician refines the symptoms
  • Specialists diagnose the patient via blackboard and using an advices of Olive Diagnostic Engine
  • Patient store diagnosis on the local machine


Fig. 2. Blackboard Usage Diagram

To follow the policy on patient data confidentiality the Blackboard keeps the session data in its memory during the session only. After the session is finished all private information is removed. During the session, the information is impersonated.

To avoid the problem of co-ownership of the patient's medical card on the period of consultation we supplied the blackboard with simple merging mechanism.

4.4. Client Architecture

Below we discuss system components and their relations. Client architecture is shown in Fig. 3 below.


Fig. 3. Olive Client Architecture Diagram

The entire functionality is produced by the interaction of different services:

It is important to note some other parts of Figure 3:

4.5. Experts Search

During the search of the most suitable expert all statistic data about consultations that have already been completed is also analyzed. Entries about consultations are stored in system database on HealthWare server and are linked with the concepts and contexts in which they were used. Analysis of this links allows making suggestions for both the doctors and patients during future consultations.

Expert search is performed on the HealthWare server. Search query is being formed automatically by the agent on the basis of user's health state. Query could be considered as an ontological model of the patient's symptoms, which could also be used in conjunction with the additional filtering request that is formed by the spec-physician.

Ontological profiles of the available experts and query are being compared during the search. If this query is performed not for the first time then already created experts contacts are being used. After the search is done, patient is able to choose the most appropriate specialist on the base of their cost and rating.

4.6. Technologies used

As for technologies used, whole Olive system is developed using Microsoft Visual Studio 2005 (C# 2.0) and is based on Microsoft .NET (Compact) Framework v.2.0. Video is captured from a web camera and transmitted using standard Microsoft Research Conference XP functionality. Desktop clients are tested on various Microsoft Windows platforms, PDA client is compatible with MS Windows Mobile 2003. We also use Web Services Enhancements (WSE 2.0) for inter-agent communication and ActiveSync 4.1 for PDA Synchronization. MS SQL Server 2005 is used as a back-end to store all information required for HealthWare operation.

5. Concluding Remarks

In such a short proposal it is hard to cover all aspects of the Olive project. There is one point to note, however: Olive is oriented not only on the medicine, but nothing can prevent it from being used in other areas where expertise is required, for example, for consultation with the sport trainer or traveling agency.

The main idea of Olive project is to investigate the process of medical consultation and automate it using AI technologies. We can stress three main points we believe make Olive powerful and successful: it helps patients to have a consultations as fast and reliable as possible by using formalized communication, it provides the way to use a fantastic power of social groups of experts, breaking the geographical and inter-lingual boundaries, and finally it lets us accumulate and use a wealth of experience of the medical scientists from all over the world, enabling us to live healthier lives using the most modern technologies!

MaiLabs © 2006.
All rights reserved.
Designed by George K,
koo.