HUME Risk Analysis
Introduction
HUME is a digital platform for detecting stress through artificial intelligence, based on changes in physiological signals related to stress. HUME provides insight into stress experiences to support caregivers in providing daily care and assistance to people.
The use of the stress detection system HUME carries potential risks. A thorough risk analysis focuses on identifying these risks and developing measures to manage or minimize them.
This document describes the possible risks based on FMEA and DPIA (Data Protection Impact Assessment) analyses and the control measures.
Risks and control measures
HUME consists of sensors and a data dashboard. The sensors measure physiological data such as a client’s skin conductance and heart rate. The dashboard then displays the stress level.
Risks
The main risks and considerations include:
Privacy, security, and ethics: Collecting data to detect stress may involve sensitive information. There is a risk that personal data may be collected without sufficient consent or that this information may be misused.
Accuracy and reliability: Stress levels are complex and can vary from person to person. The accuracy of stress detection systems may vary, and it is important to consider possible errors or inaccuracies in the results. Sensors measuring stress, such as heart rate monitors or skin conductance sensors, can be sensitive to external factors, movement, or environmental conditions, which may result in inaccurate measurements.
Stigmatization and misinterpretation: Misinterpretation of stress indicators can lead to stigmatization or incorrect treatment of individuals. It is important to acknowledge that stress is not always negative and should be understood and addressed properly.
Dependence on technology: Excessive reliance on technology to detect stress may lead to ignoring important human signals and intuition and may potentially reduce the human interaction and support needed to deal with stress.
Invasiveness and comfort: Some sensors may be perceived as intrusive, especially when they continuously collect physiological data. The sensor may also cause allergic reactions. This can reduce user comfort and lead to resistance to wearing or using such sensors.
Control measures
Privacy, security, and ethics
The following measures have been implemented within Mentech to control the risk of "Privacy and ethics":
Use of informed consent, with a transparent and clear consent procedure. Users (often via the legal representative) explicitly agree before their data is collected. This is described in our GDPR handbook and privacy policy (Privacy policy )
We store the data during HUME programs in an encrypted form so individuals cannot be directly identified via the collected information. After completion of a HUME program, the information is stored anonymously and deleted after 1 year. This is described in our GDPR handbook and privacy policy (Privacy policy )
We work according to the principle of data minimization, whereby only the necessary data relevant to the purpose of stress detection is collected. This prevents unnecessary collection of sensitive information not directly related to the purpose.
We have implemented strong security measures. Robust security protocols protect against unauthorized access, data leaks, and hacking. We use encryption, limited access rights, and secure storage methods. This is described in our security policy (Security policy )
We provide clear instructions on the impact of using HUME, such as how the stress detection system works, what data is collected, and how it is used. We also indicate how they can withdraw their consent.
We operate according to applicable standards, ethics, and legislation. We adhere to ethical guidelines for data collection and use.
We regularly perform audits (pen tests) to ensure that the collected data is processed and used ethically and legally.
We involve users (such as legal representatives and caregivers) in the process by gathering feedback and allowing them to participate in decision-making regarding the use of their data.
For questions or comments, we have set up a privacy team: privacy@mentechinnovation.eu
Accuracy and reliability
To control the risk related to the accuracy and reliability of stress detection systems, the following measures have been implemented:
Stress detection systems are not perfect and can yield false positives or false negatives. Before starting a HUME program, we provide an explanation of accuracy and tolerance limits regarding false negatives and false positives. A false positive may lead to unnecessary interventions, while a false negative may mean that someone experiencing stress remains unnoticed. During intake, we use a checklist that also discusses and defines the consequences of a possible misindication and associated interpretation of stress. In addition, during each HUME program, we collect user feedback on the accuracy of HUME. Before starting a program, we determine the baseline measurement. During model calibration, we assess whether perceived stress levels match the detected stress levels, allowing the system to be adjusted and improved. Also during use, we regularly receive feedback on the accuracy of stress prediction.
We have implemented quality control to monitor the quality of the sensor and wearables. For example, the quality of the sock is measured and assessed before the start of a session.
We typically work with 2 different sensors, namely heart rate and skin conductance, which provide both a more reliable combined signal and a built-in backup (if one sensor is insufficiently accurate, HUME falls back on the other sensor).
We have validated HUME outcome measures in a clinical study. We regularly conduct research in collaboration with academic partners such as LUMC, Tranzo, and TU/e. The results of these studies are described in scientific papers.
We use machine learning and AI algorithms to train and improve the system based on collected data. By continuously learning from new data, the system can become more accurate in recognizing stress signals. We also use lab data to develop new models.
We invest in exploring new sensor technologies and improving existing technologies. This increases the accuracy and reliability of HUME.
We work with multidisciplinary teams, where technical experts, users, and healthcare professionals collaborate to ensure and continuously improve the accuracy and reliability of stress detection systems.
Detailed information about these control measures can be found in the technical documentation of HUME.
Stigmatization and avoiding misinterpretation
To control the risk related to stigmatization and avoiding misinterpretation, the following measures have been implemented:
Stress is complex and can have various causes. It is crucial to understand the context of stress levels and not rely solely on sensor data. Implementing a contextual approach helps to interpret stress levels more accurately.
We provide HUME users with comprehensive instruction and intensive training before they start using HUME. Proper education helps prevent bias and misinterpretation of stress indicators.
During intake and training, we emphasize that stress responses differ from person to person and that not all signals mean the same for everyone.
We often combine sensor data with other information sources, such as self-reporting by the user, incident reports, or contextual data such as environment or social situation.
We emphasize that HUME stress detection is a supportive tool for providing assistance to people with care needs. We place the quality of life of both the client and caregiver at the center.
Through the MyHUME dashboard and reports, we consistently communicate about the interpretation of stress outcomes and possible appropriate interventions.
It is very important to adopt a human-centered approach when using HUME. Avoiding stigmatization starts with understanding and respecting individual differences and experiences. During intake and instruction, these risks are discussed with the care team.
Dependence on technology
To control the risk related to dependence on technology, the following measures have been implemented:
During intake and instruction, but also during the use of HUME, we discuss the role of technology in detecting and interpreting stress. We have developed comprehensive instruction and training in which we also point out the limitations of the technology to HUME users (caregivers, healthcare professionals, and relatives). During training, we teach them how to interpret data from the HUME app and the MyHUME dashboard and combine it with human observations.
The caregiver can record human assessments using labels in the app. These labels can be used to fine-tune the model to the user and are also used to provide context to the stress outcomes.
We regularly conduct evaluations to check how the technology is used and how people respond to it. We collect this feedback from users and incorporate the advice into the further development of HUME.
Mentech account managers have experience in long-term care for people with challenging behavior. They are highly skilled in interpreting behavior and can therefore highlight the importance of human interaction and support in dealing with stress. They encourage personal conversations, therapy, and social contact as a supplement to the use of HUME.
During preparation, instruction, training, use, and evaluation, we continue to emphasize that HUME is a supportive technology for recognizing stress build-up. By using technology as support rather than as a replacement, dependence on it can be reduced while maintaining human connection.
Invasiveness and comfort
To reduce invasiveness and improve comfort when using stress detection systems with sensors, the following control measures have been taken:
We design in co-creation with healthcare, e.g. by using focus groups and iterative design, smart wearables that are as non-invasive as possible and provide comfort to the user. The smart sock, for example, integrates easily into the client’s primary process. The ankle heart rate sensor has also been designed for client comfort. We have paid great attention to user-friendliness during wearing.
During the baseline measurement, we conduct a comfort test and an acceptance test. We look at both the acceptance of the sensor and the comfort (e.g. does the sensor cause an allergic reaction). Also during use, attention remains focused on comfort and acceptance by regularly discussing this topic with care teams.
At the start of a HUME program, we assess the risk of swallowing. We also try to estimate whether there is a chance of getting caught on something.
We use different sizes of sensors (socks) and different types of sensors to offer a personalized and tailor-made solution.
By focusing on improving comfort and reducing the invasiveness of sensors, the acceptance and use of stress detection systems can be increased without users feeling uncomfortable while wearing them.