The US Department of Defense (DoD) recently completed a project exploring how artificial intelligence (AI) chatbots might support medical services in the military.
Known as the Crowdsourced AI Red-Teaming (CAIRT) Assurance Program, the initiative focused on testing large-language models (LLMs) for tasks like summarizing clinical notes and providing medical advice to service members.
Led by the technology nonprofit Humane Intelligence, the program involved over 200 independent participants, including healthcare providers and analysts.
Did you know?
Subscribe - We publish new crypto explainer videos every week!
What is Polkadot in Crypto? (DOT Animated Explainer)
As the DoD noted in a January 2 announcement, the program will produce benchmark datasets to guide future evaluations of similar technologies and vendors.
Teams from the Defense Health Agency and the Uniformed Services University of the Health Sciences also joined to evaluate the technology. They examined three chatbot models to identify weaknesses and potential risks.
The DoD's pilot revealed over 800 issues after using chatbots in military medical applications. These issues include "potential vulnerabilities and biases".
Matthew Johnson, who oversees the initiative within the DoD’s Chief Digital and Artificial Intelligence Office (CDAO), highlighted that the findings will influence the department’s future research into generative AI. The goal is to ensure AI tools meet the performance and security standards required for military use.
The CDAO, established in 2022, plays a key role in advancing AI and digital technologies within the US military.
Meanwhile, the United Kingdom recently sought feedback from the AI and creative industries as part of a consultation on new rules. What does the government want to address? Read the full story.
