Big Data For Big Wars: JEDI vs. China & Russia

WASHINGTON: The military needs the controversial JEDI cloud computing program to wage “all-domain warfighting,” especially “in the Pacific,” the three-star chief of the Pentagon’s Joint Artificial Intelligence Center says. Without a common system to share data rapidly among units in different services and different theaters around the globe, the Defense Department cannot implement the high-speed, AI-assisted Multi-Domain Operations it considers essential in future conflict against advanced adversaries.

“This is not merely an AI issue,” Lt. Gen. Jack Shanahan said. “It is also about joint all-domain warfighting, taking advantage of emerging technologies to develop new operating concepts for a kind of warfare that will look completely different than what we’ve experienced for the past 20 years.”

Shanahan spoke alongside Pentagon CIO Dana Deasy at a closed-door media roundtable. The conversation and the coverage were dominated, understandably, by reporters’ questions on President Trump’s public complaints about the JEDI program and the prospect of a “pause” while Defense Secretary Mark Esper reviews it. In brief, Deasy said there is no “pause,” but the contract award won’t come this month as originally hoped — both points we had already made a week earlier.

But why did Deasy bring along Shanahan, who has no role in the JEDI contract process? “I’m as far removed to that as I could possibly be,” the general said. The AI chief’s inclusion was clearly an attempt to change the topic from the troubled competition to what the Pentagon thinks really matters most.

The US military has made major strides in applying Artificial Intelligence to military intelligence, most famously with Project Maven, which Shanahan led before he was tapped to stand up the year-old Joint AI Center. But both Maven and JAIC’s own projects require tremendous amounts of human labor to set up and keep updated, Shanahan said, because of the bureaucratic and technical obstacles to sharing data.

Data, not only in vast amounts but tidily organized in compatible formats, is essential to train and use machine learning algorithms. In the military, that means not only secret intelligence data, but reams of mundane planning information about supplies, mechanical breakdowns, and troop movements.

Sure, lots of military organizations have moved their data to “the cloud” — in essence, to centralized servers accessed over a network — but they’ve moved them to different clouds with different data formats, access procedures, and authorized users. The Defense Department currently spends about half a billion dollars a year on roughly five hundred different cloud contracts. Without a single, common “enterprise” cloud, you can’t automatically share data amongst different users, let alone distribute cybersecurity patches and new software automatically.

So Project Maven started out “sending a crew site by site-by-site to install that first algorithm, but even worse, having to go back site-by-site to then put updates into that,” Shanahan said. Even today, he’s told, “one of the task forces in a different location is bringing enormously important information and data, but there’s no simple way to share that with another part of Afghanistan or another part of the task force.

“The pace of operations in Afghanistan … it’s higher than it’s been in an awful long time right now,” Shanahan said. “Imagine the speed of operations in a fight in the Pacific, where you just do not have time to figure out ‘how do I get my data, clean my data, move it from point A to point B?'”

“The whole idea of joint all-domain command and control — which the Joint Staff J-6 [i.e. communications branch] has now been chartered to lead … is going to rest on the backbone of a, you know, enterprise cloud solution,” Shanahan said, “so different fleets can communicate, different air forces, Air Force communicates with [Navy] fleets, communicates with [Army] divisions.”

“In this future high-end fight, we envision a world of algorithmic warfare and autonomy where competitive advantage goes to the side that understands how to harness 5G, AI, enterprise cloud, and quantum, when quantum’s available, into a viable operational model,” Shanahan said. “It’s all part of the department’s transformation …. to an all-domain digital force.”

How We Got Here: Beyond The Buzzwords

What Lt. Gen. Shanahan said Friday about JEDI is just the tip of an iceberg that’s been growing under the water for years. Largely out of public view, new ideas and technologies are crystalizing, under pressure from rising Russian and Chinese threats, into a new American way of war.

Known as Multi-Domain Operations, the new concept requires the Army, Air Force, Navy, and Marines to work more closely together than ever before, at an ever-faster pace, against ever more sophisticated adversaries. If US jets can’t get past anti-aircraft defenses, for example, maybe a ground-based hypersonic missile can blow up the enemy’s launchers, or military jammers and hackers can disable their radar, allowing the air strike to proceed. This level of coordination, in turn, requires not only sharing vast amounts of data rapidly amongst the services and between theater commands, but Artificial Intelligence to help make sense of it in time to act.

This new approach of war was born in the Obama Administration under former Deputy Defense Secretary Robert Work and the outgoing Chairman of the Joint Chiefs of Staff, Gen. Joseph Dunford. It has gained traction under Trump with the patronage of the Chiefs of Staff of the Air Force, Gen. David Goldfein, and of the Army, Gen. Mark Milley, whom the president has tapped to be the new joint chiefs chairman.

To understand where this is going, you need to understand where it came from — which Breaking Defense has covered from the beginning:

• 2011: Even as the Army and Marine Corps “surge” forces into Afghanistan, the Air Force and Navy refocus on the rising threat of China. While the US hunted low-tech insurgents and terrorists, Beijing has built an increasingly sophisticated arsenal to keep American forces at bay. To defeat this “Anti-Access/Area Denial” defense, the Air Force and Navy envision a new collaboration called Air-Sea Battle.
• 2014: Russia seizes Crimea in February and invades eastern Ukraine, reviving the risk of major power war in Europe. By November, Defense Secretary Chuck Hagel — advised by Work — launches the Third Offset Strategy to counter Russian and Chinese advances with new technologies, above all AI.
• 2015: Gen. Dunford says the Joint Staff needs new capabilities to handle metastasizing trans-regional threats that bleed across the traditional bureaucratic boundaries between geographical combatant commanders. That means sharing data between headquarters around the world.
• 2016: Finally refocusing from counterinsurgency to the great power threat, the Army announces a new warfighting concept called Multi-Domain Battle, now known as Multi Domain Operations. Ground troops will support air and sea forces against advanced adversaries, the services working together closely across the five officially recognized domains of war: land, sea, air, space, and cyberspace.
• 2017: The Air Force begins to talk publicly about their own parallel effort, Multi-Domain Command & Control, to develop the technology and organizations required to actually integrate operations across all five domains. Deputy Secretary Work announces a near-term application of Artificial Intelligence to military operations, the Algorithmic Warfare Task Force — the parent of Project Maven and forerunner of the Joint AI Center. And the Pentagon starts soliciting industry input on the cloud program that will become JEDI, the Joint Enterprise Defense Infrastructure.
• 2018: Defense Secretary James Mattis releases a new National Defense Strategy explicitly saying China and Russia are the primary threats. As the Army collaborates more closely with the other services, it expands the tactical, combat-oriented Multi-Domain Battle to the broader concept of Multi-Domain Operations. Pentagon officials create the Joint Artificial Intelligence Center under CIO Deasy and formally launch the competition for JEDI, describing it as an essential tool not just for high-level headquarters but also for frontline troops — but it soon runs afoul of congressional concerns about fair play and conflicts of interest.

In many ways, the fight over these new ideas is an old and painfully familiar story: Innovative ideas and technology run aground on bureaucracy, vested interests, and either actual corruption or the paralyzing fear of potential corruption. Meanwhile, the troops don’t get what they need.

JEDI may not be the solution. The current contract structure may not be optional. But the military needs something to do the job, and soon.

“The warfighter needed enterprise cloud yesterday,” Shanahan said. Without enterprise cloud, there is no AI at scale: AI will remain a series of small-scale stovepipe projects.”

Meanwhile, “our potential adversaries are doing it at their own speed,” he warned. “Whether it’s Baidu, Alibaba, Tencent, SenseTime, they’re all coming up with their own cloud solutions and — and then, don’t get me wrong, I don’t want to make them 1,000 feet tall — they’re going to have their own cloud interoperability challenges. But [given] the level of investment and the number of people they’re putting at the problem, they’re moving at a very rapid pace, and what I can’t afford to do is slow down anymore.”