Challenges to Regulating AI Models Used in Biomedical Research

November 18, 2024 Dennis D. McDonald

The issues raised by the need to regulate the use of AI in biomedical research, as reviewed in AI and Biosecurity: The Need for Governance (Science, August 23, 2024), serve as a stark reminder that our inability to accurately predict the future can lead to significant and potentially deadly risks.

The core challenge is preventing AI-based modeling and analysis from being used to create lethal and uncontrollable consequences, such as the development of deadly, easily spread pathogens.

Biomedical researchers are no strangers to managing risk. For instance, in the United States, rules and legally enforceable regulations already govern the treatment of human and animal subjects. Similarly, regulations exist for certain "dual-use" research—work that has the potential to yield both beneficial and harmful outcomes.

Despite these safeguards, advancements in AI modeling, even when aimed at achieving positive treatments and outcomes, also heighten the risk of synthesizing harmful pathogens.

The challenges of regulating the use of AI models in sensitive biomedical research are formidable:

Uncertain outcomes: Research inherently involves unpredictability, no matter how meticulous the planning.
Dual-purpose potential: The same AI-based models can be used for both beneficial and malicious purposes.
Diverse research environments: Not all potentially risky research in the U.S. is federally funded or managed; some is conducted by private sector organizations.
Global coordination: Effective regulation would require the participation of multiple nations, including major players like the United States, China, Germany, and the United Kingdom.

At first glance, these regulatory challenges seem to me insurmountable. For instance, achieving international consensus on rules governing AI use in biomedical research is identified as a critical hurdle, but given the potential for political instability here in the United States, how likely will the US be willing to play a leadership role? Still, the Science article highlights some examples of progress, and the risks of failing to regulate the use of AI in this field are too grave to ignore.