Blockchain, big data and genAI: US Army uses novel tech to track billions in Ukraine aid

WASHINGTON – With foreign policy in flux, new Army contracts in question, and humanitarian aid frozen, including to Ukraine, there’s never been more pressure on the Pentagon to prove that military aid to Kyiv — almost $70 billion to date — is being managed efficiently and accountably.

The lion’s share of that burden falls on Army Materiel Command which oversees everything from arsenals building cannon barrels, to warehouses of equipment being transferred under Presidential Drawdown Authority, to an international distribution network spanning private companies, NATO allies, and frontline Ukrainian brigades.

American logisticians are rightly famous for their ability to move “iron mountains” of equipment around the world. But their proficiency at keeping track of it is patchy, as the Pentagon’s own Inspector General has pointed out in several recent audits of aid to Ukraine. So making sure the iron mountains arrive in good condition at the right place and time requires managing a mountain of paperwork. Nowadays, most of that data is virtual — but hardly all.

“Inside of the Army, I would say near 100 percent of that data is automated,” said Chris Hill, AMC’s chief data and analytic officer. That is, he told Breaking Defense in an interview, the information is automatically, digitally transmitted from database to database without needing a human to laboriously re-key it.

But everybody else’s data? Not so much.

Kyiv’s post-Soviet institutions are still struggling to implement modern business process automation services such as SAP, said Lt. Col. Colby Smithmeyer, a senior AMC analyst. So, he told Breaking Defense, “data we’re getting from the Ukrainians, it could be coming from a phone call from a liaison officer, it could be an Excel spreadsheet in digits, or it could be a printed-out Excel spreadsheet.”

Standard Defense Department data systems were never built to tame such a mad menagerie of different sources. So soldiers and civilians across AMC have innovated, improvised, and on occasion outright kludged together a digital toolkit including big-data analytics, generative artificial intelligence, and, in their most recent experiment, blockchain.

“It’s pretty important we maintain visibility, accountability, and auditability of those specific pieces of equipment,” said Pat Sullivan, a civilian Highly Qualified Expert at the command, in an interview with Breaking Defense. “We have to maintain that full end to end [visibility] in a contested environment, [so] we can see it and trace it and track it all the way through.”

RELATED: Telemaintenance and stockpiles: Army Materiel Command takes its own lessons from Ukraine

From American Warehouses To Ukrainian Frontlines

Any given combat vehicle, box of ammunition, or pallet of supplies may begin its journey in one of AMC’s own arsenals, another government agency’s warehouse, or a contractor’s factory, officials explained in interviews. Then the aid has to be handed off to the military transport units and/or private shipping companies that physically move it overseas, overseen first by US Transportation Command, which handles inter-theater movements, and then by US Army Europe and Africa, a four-star theater command. Only then does it reach the Ukrainians.

As a shipment moves through the system step by step, each of those actors needs to be able to see what’s coming to them next. Then, once they receive it, they need to update everyone else that it’s arrived intact and in the proper hands, plus when and how they’ll send it on. So AMC needs a system where all the authorized actors — but no one else — can read and modify the data relevant to them — but not anything else.

To track all these transactions securely, AMC is testing a technology best known for cryptocurrency: blockchain. Like crypto, after all, information on military aid needs to be accessed securely by authorized users around the world who don’t share a single, centralized database (and who may not entirely trust each other).

“It proves traceability by creating an immutable record of transactions that allows DoD to track these assets,” said Maj. Matthew Goyette, who led a blockchain pilot exercise. “It does away with all of the manual processes, the Excel sheets, the PowerPoints, the calls, the emails. The data is cryptographically hashed into that ‘block’…for select parties to see at the exact level you want them to see it at.”

Officials declined to say when the pilot began and how long it lasted, but they did say AMC tapped expertise on blockchain from a host of partners in and out of government: Defense Logistics Agency, Air Force Research Lab, public-private digital manufacturing hub MXD, and contractor SIMBA.

But they couldn’t just copy-and-paste existing commercial solutions, which notoriously consume a lot of processing power and electricity, Sullivan and Goyette emphasized. Instead, they built a slimmed-down blockchain that only stored encrypted keys to the logistics tracking data, rather than putting all the files in the blockchain itself.

What’s next? “We’ve proven out the initial pilot and we’re determining with leadership how we scale [up],” Sullivan said. The technology’s future at AMC will depend on finding funding, he acknowledged. But he’s optimistic because the command has been investing resources and high-level attention into a lot of digital projects, of which blockchain is just the latest one. Much of that investment has been driven by the demands of suppling Ukraine.

Ukraine ‘Lit a Fire’

With its vast logistical responsibilities, Army Materiel Command has been keenly interested in better data management for years. But “once the Russians invaded [Ukraine], it really lit a fire under this,” Hill said.

“We went over there with algorithms that were not really ‘AI.’ I would call them ‘advanced analytics,’” he told Breaking Defense. “We decided we’ve got to bring more AI in this, [and] about a year ago, we started really getting into the generative space.”

While chatbots are infamous for “hallucinating” made-up information, intensive study reassured the Pentagon that GenAI can be constrained to cite only authoritative government data, instead of random Reddit posts about putting glue on pizza. Once suitably tamed, Large Language Models show a remarkable ability to digest and organize what’s called “unstructured” data: information that’s interspersed through a passage of ordinary English text, rather than carefully categorized and labeled in a “structured” database.

This capability comes in especially handy for AMC, because the Presidential Drawdown Authority orders specifying what aid to give Ukraine are, in fact, unstructured. Instead of sending out precisely formatted files that load seamlessly into military logistics databases, the White House issues PDFs.

At first, Army personnel simply had to sit down, read through every order line by line, and hand-type all the relevant information into the proper fields in AMC databases, explained Lt. Col. Joseph Lavalle-Rivera. But manual data entry is notoriously labor-intensive, slow, and prone to error, and the drawdown orders were just too big, and the schedule too tight, to do it all by hand.

Even in early orders, Lavalle-Rivera said, “you would have 200 items with 12 to 15 different fields associated with each.” That’s easily over 2,500 individual entries per drawdown order — and they just kept growing.

Next AMC tried to automate the data entry by writing traditional, non-AI code to search the drawdown orders, extract the data, and upload it. This worked better but only briefly, because traditional IF-THEN-ELSE software struggled to parse even minor formatting changes from one PDF to the next.

“The format constantly changes, and every time they change the format, add a new piece of data, or whatever... it immediately breaks [the code],” Lavalle-Rivera told Breaking Defense. “Even changes of an additional tab, extra lines, can throw things like that off.”

So rather than constantly hand-write and rewrite code to parse the PDFs, AMC turned to GenAI.

At first that didn’t work well either. On the upside, the LLM didn’t “hallucinate” and add made-up data, because it was restricted to answering based on what was actually in the official documents. On the downside, it often omitted data that was actually there, because the algorithm initially interpreted items marked “classified” as things it wasn’t allowed to tell its user about.

It took Lavalle-Rivera “about a week of messing with it” before the LLM would give a full and accurate answer, he said. “A lot of that comes down to the prompt engineering. … There’s more of an art to it than you’d think.”

Predictive Data For Future Fights

Once the LLM was tamed, and its outputs double-checked, it has provided an efficient way to turn unstructured Presidential Drawdown Authority PDFs into structured data that’s digestible by Army Materiel Command’s big-data system.

That system is a variant of the Army-wide Vantage platform, but modified to allow easier input of non-US data from partners like Ukraine. The command calls it APAS, the AMC Predictive Analytics Suite — “predictive” because it not only tracks the flow of arms and supplies but tries to forecast what Ukraine (or US units) will need next.

“If you need ammunition and you’re in eastern Ukraine and you have to go through the process of requesting that, getting a Presidential Directive, having that get funded, acquiring the ammo, shipping it, etting it there — that’s quite a long timeline,” Hill told Breaking Defense. “[Hence] the importance of predictive analytics: I’ve got to know when I’ve got a problem a couple of months out so I can work that now.”

The US Army is learning a lot about the challenging logistics of modern war as it supports Ukraine, added Smithmeyer.

Starting out, he said, “we didn’t have a good idea for a lot of these consumption [rates] because we didn’t have a good baseline.” Take the rate at which M777 howitzers wore out their gun barrels, he offered by way of example. “The rate at they were firing was unlike anything we have seen certainly in Iraq and Afghanistan.”

Ukraine is pouring ever greater quantities of real-world battle data into Army Materiel Command’s analytics, Smithmeyer said.

“Now we’re tracking the maintenance status of the [Ukrainian brigades in theater, we’re tacking the consumption of supplies in theater,” he told Breaking Defense. “[That] is building a lot of data sets that now we can use…As we collect more and more data, we’re able to get better and better predictions.”