This editorial appears in the September Issue of the American Journal of Bioethics
Liougas et al.’s composite case study analysis of real-time location systems for dementia care in a residential care home setting highlights the ethical problems that follow when a technical system is introduced without a clear and collectively shared purpose. Despite the growing consensus that decisions about technology design and implementation should be informed by the perspectives of people with dementia and caregivers, in the case study, neither tool developers nor care facility leadership envision dementia patients and caregivers as vital sources of input; in fact, they do not appear to have consulted caregivers or people with dementia. As a result, care facility leaders and other stakeholders were poorly positioned from the outset to determine whether the technology supported the needs and priorities of patients and caregivers. While concerning in any dementia care setting, the absence of patient and caregiver perspectives becomes all the more consequential in the context of AI-enabled technologies. On the one hand, AI practitioners and dementia researchers describe the potential of AI to transform dementia care—for example, by supporting clinical decision-making and differential diagnosis, promoting earlier detection and care planning, and facilitating remote monitoring. On the other hand, AI’s rapid development and capacity to reshape established boundaries and norms risk outpacing our collective ability to perceive its longer-term consequences.
A rigorous approach to AI ethics in dementia care therefore requires sustained engagement with patients and caregivers throughout the technology lifecycle. Such engagement can help resist epistemic injustice by countering the systemic devaluation of the knowledge and expertise of people whose perspectives are marginalized. A user-engaged approach is also consistent with recommendations that people with dementia participate in technology design and research, either independently or with caregiver support, to help ensure that tools reflect and respond to patient and caregiver needs. Moreover, Wang et al. identify a recent increase in the inclusion of people with moderate- and later-stage dementia in technology design processes. Efforts to establish best practices for co-design—that is, designing AI tools in sustained collaboration with users—in dementia contexts remain nascent. Despite calls to involve people with dementia throughout the AI lifecycle, their participation in technology design and research often remains limited to one-off consultations or user testing sessions rather than sustained engagement.
We recognize that the issue of how to elicit and understand input from dementia patients is by no means a settled matter. That is, drawing on dementia patients’ input to inform the design and implementation of dementia care technologies is at once ethically essential and ethically fraught. Accordingly, in what follows, we: 1) offer a practical overview of what a user-engaged ethics approach to designing and implementing dementia care technologies might look like across multiple stages of the technology lifecycle; and then 2) explore how theoretical uncertainty about interpreting technology user input in the context of dementia care poses significant challenges, but should not dissuade us from pursuing the vital work of engaging the perspectives of dementia patients and caregivers.
USER INVOLVEMENT ACROSS STAGES OF TECHNOLOGY DEVELOPMENT
Figure 1 describes how prospective users with dementia and caregivers can be engaged at 3 stages of the AI development life cycle. The figure briefly highlights considerations pertaining to value elicitation and interpretation that will be explored more fully in the following section.
Figure 1. User involvement across stages of AI tool development for dementia care.
Adapted from Dankwa-Mullan et al.
Display full size
Identify or Reassess Needs, Values, and Desired State of Care
Technology design, like care decisions, should be aligned with user values and priorities, requiring engagement with people with dementia and caregivers to understand their own values and priorities, as well as any overlap or tension between these values. Beyond brainstorming more efficient and cost-effective alternatives to existing modes of care, engaging prospective users from the earliest phases of design provides an opportunity to think creatively about if and how technology could be harnessed to promote autonomy and connection: for example, devising a technological tool that facilitates connection between people living alone with dementia and human carers, companions and communities. Additionally, a human-centered (rather than a technology-centered) approach would include entertaining the possibility that a lower-tech solution may be more appropriate to needs and values, keeping in mind that anticipated time and cost savings may not always materialize in real-world dementia care settings.
Iteratively Develop AI Tool
AI co-design involves iteratively refining tools in response to input from prospective users, especially for users with dementia who may have needs for comfort and accessibility that AI developers without cognitive impairment may not anticipate. Also, eliciting input on an early prototype is an opportunity to identify misalignment or resonance between the values of caregivers and of people with dementia. For example, through a participatory design process, one team learned that a safe walking device resonated with both the value of autonomy to people with dementia and caregivers’ commitments to safety and risk reduction. Proponents of co-design centering people with dementia also stress the importance of allowing people with dementia to test a proposed tool in their everyday routines and inform refinement of tool design, recommending supportive measures such as keeping co-design and testing sessions short to minimize fatigue, including facilitators who can promote understanding between stakeholder groups, ensuring adequate transportation, and taking time to build a comfortable, lighthearted atmosphere among participants.
Implement Tool and Monitor its Performance
To promote ethical implementation of AI tools, input from people with dementia could inform the design of consent/assent processes and help craft appropriate explanations of tool purpose and function. Additionally, ongoing monitoring is especially vital in AI tools for dementia care. The performance of AI tools may decline over time as inputted data increasingly diverges from the data on which the tool was trained. Users’ cognition will also change over time. A broadened conception of monitoring would go beyond technical evaluation and maintenance to include ongoing consultation with users to see if the tool serves evolving needs, if new adaptations or supports are needed, or if the tool is no longer appropriate. Such a process would offer people with dementia opportunities to exercise agency regarding their care beyond the narrow scope of consent and assent.
INTERPRETIVE CHALLENGES AND EPISTEMIC INEQUITIES
Co-design, while vital, has the potential to reinscribe the epistemic inequities that it is often designed to subvert. Critical voices in the field of neurotechnology highlight ableist assumptions and systems that can undergird, or be reinforced by, the extraction of user feedback without concrete benefit, lack of accountability for tech-based harms, subtle devaluing of “lived experience” expertise and failure to adapt community engagement processes to the needs of users and advocates.
Dementia may introduce additional challenges to harnessing co-design to promote epistemic justice. One such challenge pertains to the interpretation of user values. It is important to note that expressing our values is difficult even without cognitive impairment; one reason for this is that values may be so embedded in our sense of the world and ourselves that it may sometimes be difficult to bring them to conscious awareness or express them in language. People with dementia will struggle more than most to articulate their values verbally, and nonverbal expressions of emotional states like distress are open to interpretations ranging from mere noncompliance to principled objection. Interpretations in either direction, if not tempered by humility and genuine curiosity, may produce the kind of erasure of people with dementia’s perspectives that commitments to both epistemic justice and technology co-design resist. Tendencies to embrace one extreme or the other recall anthropologist Lawrence Cohen’s concept of “ironic listening”. Ironic listening means attending to people with dementia without either reducing their expression to mere “babble” or seeking the deep, hidden truth in their statements or actions. It evokes the hard work of self-reflection and attunement that is required to avoid either over- or under-determining meaning.
We are navigating this interpretive challenge and reflecting on our own epistemic authority in our ongoing research, which aims to elicit the personal values of people with dementia, explore if or how values change over time, and understand the impacts of potential value change on caregivers. We endeavor to investigate the impact of neurogenerative disease on valuing without recourse to ableist frames that conflate divergence from normative modes of expression or social engagement with loss of or disengagement with values. There is no step-by-step guide to ironic listening. But in our experiences interviewing people with dementia and their caregivers, often in their homes, we have found it useful to engage with people with dementia through multiple communicative modalities and affective registers. For example, we learn about treasured objects and attend to moments of joy, which can often be indicative of values.
In technology design and implementation, institutional incentives may make such open-ended, time intensive value elicitation difficult. Both community engagement and value exploration take time. Indeed, accounts of co-design involving people with dementia emphasize the importance of building comfort and an atmosphere of ease to ensure a positive experience for dementia patients with varying needs. This may be misaligned with the “move fast, break things” ethos of technology and AI innovation. Even beyond private industry, public institutions and academic medical centers often face pressure to push innovation, and ethical deliberation may be constrained, de-emphasized or reduced to legal compliance.
The design and implementation of digital tools may also expose value misalignments between people with dementia, caregivers, healthcare professionals, and developers. A newly developing literature on community-engaged health AI development provides practical guidance for eliciting multiple stakeholders’ values regarding a tool, but guidance for adjudicating value conflicts is lacking. It is difficult in any domain to adjudicate value conflicts without conferring preference to those who possess more conventionally legible forms of expertise or institutional status. Cognitive impairment from neurodegenerative disease, and resulting communication difficulties coupled with interpretive challenges, may amplify the temptation to privilege technical considerations or institutional priorities over user concerns.
AI co-design in dementia will pose serious interpretive and conceptual challenges. The problem of responsibly interpreting the verbal statements and nonverbal expressions of people with dementia carries particular ethical weight, as even well-intended efforts to amplify their voices may instead inadvertently silence them—a danger we recognize and continue to grapple with in our own research. Still, in our view these challenges cannot be evaded if new digital technologies are to fulfill their promise to improve care for people with dementia and their caregivers in ways guided by their own values.
Juliana Friend, PhD, Valerie Black, PhD, and Winston Chiong, MD, PhD