Does Pentagon need an overarching AI acquisition plan? A key official is wary

WASHINGTON — While the Pentagon is spending billions of dollars on artificial intelligence, the deputy in charge of the office responsible for the acceleration of data and AI adoption said that it might not be necessary to have department-wide guidance to inform its AI acquisitions. 

During a RAND Corporation event on Tuesday, Margaret Palmieri, deputy chief digital and AI officer (CDAO), said the office is “not far along in providing department AI guidance, but I don’t know if we necessarily need to.” 

“What we’ve really been trying to wrestle with inside of CDAO is not to over centralize because the department is so diverse and distributed and so large, that we want innovation to happen at the edge,” Palmieri said. “And so we think we have some best practices, but not necessarily ready for blanket policy guidance quite yet.”

A recent report by the Government Accountability Office tasked the CDAO with spearheading the effort to develop a department-wide strategy in hopes of preventing the DoD from spending money on technologies that don’t address future challenges raised by AI-enabled adversaries. The GAO also stated that each military service should develop their own guidance to help navigate the acquisition process. 

Palmieri noted that the CDAO already has core acquisition vehicles, like Tradewind, that work to accelerate AI adoption. The platform is focused on increasing speed at which AI technologies are adopted and connecting innovators, but the CDAO is “not really on the path for policy quite yet,” she added. 

The Pentagon has asked for $1.8 billion in its fiscal 2024 budget request for various AI efforts and has established initiatives like AI and Data Acceleration that aims to improve tactical AI at combatant commands. DoD has already also released several AI-focused strategies, like its Responsible AI Strategy and Implementation Pathway and Ethical Principles for AI. 

Meanwhile, Palmieri said the CDAO is experimenting with different generative AI models through the recently relaunched Global Information Dominance Experiments. Five different generative AI models were examined in the most recent GIDE to test how they work and if they can be trained on DoD data. 

But while the Pentagon experiments with generative AI, more attention needs to be paid to the potential downsides of the tech, particularly when it comes to “hallucinations,” which is a “huge problem for DoD,” Palmieri said. The CDAO is looking to work more closely with industry on these kinds of downsides.

“Every every operational system we deploy into the field, we train the users on how to use it,” Palmieri said. “It’s a key aspect of delivering a capability in DoD, and AI is going to be no different. And so in order to be able to train someone on how to use it, you have to know what it’s good at and what it’s not good at and when it is appropriate to use.

“And so as we think through both the scaffolding pieces and the deployment pieces, making sure that that’s part of the conversation, we don’t expect to centralize that in the CDAO, but making sure that others are thinking through that is really important,” she continued. 

Palmieri’s concern echoes comments made by retired Lt. Gen. Jack Shanahan, founding director of the Joint AI Center from 2018 to 2020 (the organization was subsumed into the CDAO in 2022), who told Breaking Defense in April that the “idea of hallucinations is a major problem.” Shanahan’s successor, retired Marine Corps Lt. Gen. Michael Groen, agreed with his comments and said the Pentagon should cautiously experiment with generative AI.