Digital patient portals present an opportunity to improve patient health, as well as the health care experience. Research suggests that they are able to foster greater engagement among patients and empower patients to participate more fully in their own care. Yet reliance on digital tools can also exacerbate disparities when minoritized groups and non-English-speakers, who already face an excessive burden of chronic disease, have lower rates of access. And the added burden of work that patient portals represent can also increase clinician burnout.

Kevin B. Johnson, MD, MS, FAAP, FAMIA, FACMI—a Penn Integrates Knowledge professor who teaches Digital Health in the Master of Health Care Innovation—has been working with colleagues to find innovative solutions to close the “techquity” gap without further overwhelming clinicians.

Johnson and colleagues define “techquity” as an inclusive and equitable online experience for historically marginalized groups. And in a 2023 editorial in JAMA Health Forum, they suggest five strategies to help achieve it:

• Develop useful metrics to evaluate access gaps.
  Organize existing data from the electronic health record in a way that is actionable, and that minimizes the effects of data collection bias.

• Encourage more equitable approaches to facilitating access.
  Help clinicians recognize and remediate biases that may prevent them from offering access to members of minoritized groups.

• Educate patients.
  Create guides and interactive orientations to the digital portal technology that are appropriate to various levels of technological literacy.

• Eliminate barriers.
  Use a human-centered design approach to create a smooth, culturally appropriate user experience that accounts for privacy, confidentiality, and other concerns.

• Make the ecosystem of virtual care more robust.
  Understand the consequences of increased utilization, barriers to proxy use by patient caregivers, and new virtual care practices that grow out of the digital portal.

Taken alone, closing the “techquity” gap could intensify the burden on physicians and make the burnout crisis worse. Johnson is addressing this challenge in collaboration with a group of undergraduate researchers from Penn’s School of Engineering by exploring avenues to automate some responses to physician in-basket messages.

This project evaluates the effectiveness of automated responses to patient messages built on OpenAI’s GPT-4 conversational language model. Johnson and his team tested two approaches to prompt engineering in order to generate response messages to patients:

• Zero-shot learning, where the AI is asked to produce a response without priming it with extensive examples and guidance.

• Few-shot learning, where the AI is given several key elements for consideration and is then asked to produce a response.

Finding better results with the few-shot approach, they developed a response-generation pipeline that accounts for patients’ literacy level, the urgency of the message, and responsiveness to user inputs.

The results of the experiments were distributed to a panel of health care providers, who were asked to evaluate them along four dimensions:

• Perceived empathy

• Relevance

• Medical accuracy

• Readability

Among the 49 providers who responded, GPT-generated results equaled or outperformed human responses across all key characteristics—and with statistical significance in the categories of readability and perceived empathy. This is especially impressive, the research team suggests, because health care professionals are more “accustomed to the nuances of patient communication” and therefore “may have higher expectations regarding message quality and relevance.”

These results are a first step. Revisions and further testing are needed to improve the response pipeline and mitigate biases and inaccuracies. And additional work is needed to understand potential ethical and liability considerations related to message automation.

However, initial experiments in AI-enabled virtual patient interactions have shown promise. And given the burdens on physicians related to digital patient portals—as well as the potential of “techquity” efforts to compound that work—projects like this serve as a necessary starting point to better serve the needs of patients while maintaining a sustainable, humane work environment for physicians and their teams.

Learn more about Kevin B. Johnson’s approach to evaluating and implementing digital tools in his course, Digital Health.