Members of the 56th Air and Space Communications Squadron at Joint Base Pearl Harbor-Hickam operate cyber systems using a Enhanced communications flyaway kit during the Global Information Dominance Experiment 3 and Architect Demonstration Evaluation 5 at Alpena Combat Readiness Training Center, Alpena, Michigan, July, 12, 2021. (U.S. Air Force photo by Tech. Sgt. Amy Picard)
WASHINGTON — In recent days both Pentagon CIO John Sherman and the House Armed Services Committee have pushed new policies to speed the adoption of commercial software by the Department of Defense.
That’s great as far as it goes, DoD and industry officials said at a recent conference. But, they argued that beyond cutting red tape, the chronically overworked officials certifying commercial software as cybersecure and safe for government networks also need the technical tools and computing environments to test the software properly.
The critical choke-point is a process known as ATO, or Authorization To Operate. When the Pentagon wants to use some commercial software, a government Authorizing Official (AO) must formally approve it as sufficiently safe and secure against cyberattack to be used on government networks. That process can be fraught with bureaucratic hurdles.
Both Sherman’s new policy and HASC’s draft legislation focus on cutting red tape, specifically by requiring authorizing officials for different DoD networks to accept each other’s authorizations, instead of requiring redundant checks on the same software — a principle known as “reciprocity.” (There are logical exceptions: For instance, software granted ATO on an unclassified network still needs additional checks before being allowed on a classified one.)
However, ATOs are not just a bureaucratic process but a highly technical one. Someone has to be able to actually check the code works properly and is secure. The best way to do that, experts say, is by stress-testing the program in a self-contained computing environment, one that works just like a real DoD network, so the testing is realistic, but which isn’t actually connected to anything sensitive. Such enclaves can provide a kind of playground — or sandbox — where developers roll out new code, experiment with it, get feedback from actual users, and rapidly refine it into new software or updates for applications already in use.
But this computing infrastructure takes time, money, and expertise to set up, so its availability can become another bottleneck, experts said at the Offset Symposium conference Thursday.
“Software modernization over the last 10 years is night and day, [but] the ATO process is still hard,” said Donald “Chee” Gansberger, a software developer and Afghanistan veteran who now works at AFWERX, the Air Force’s outreach arm to innovative private companies. “There’s a lot of memos that have just come out and a lot of policy changes, in literally the last month, that are trying to change this, or going to come in the very near future.”
On the technical side, Gansberger continued, there’s a growing number of software factories using “agile” software development methodologies to develop code more rapidly and then keep it continuously updated. Some of these islands of excellence are inside DoD, he said, but there are also private firms that have set up high-tech systems specifically to test commercial software on DoD’s behalf as part of the ATO process, such as Second Front’s Game Warden. (Second Front also sponsored the Offset conference).
“Onboarding commercial partners is really where we’re seeing a lot of value,” Gansberger said. “Without tools like Game Warden or other software factories, onboarding [them] would have been impossible.”
Better infrastructure to test and experiment with new software is definitely making a difference, agreed Steve Escaravage, head of AI and analytics at Booz Allen Hamilton. Until recently, he told the conference, “the thing that’s preventing adoption of emerging technology [was] always the same thing: It’s access to accredited enclaves or computing environments, access to data, and then access to operational warfighters or folks in the mission to give their feedback. We have some solutions now… I’m actually fairly optimistic, while there’s still hurdles, we’re making great progress there.”
Not all Authorizing Officials have easy access to such high-tech tools, however, said Derek Strausbaugh, who leads the DoD Mission Team at software giant Microsoft. “Excel spreadsheets and Word documents … that’s not the way to authorize a system,” he told the conference.
“I see AOs on most software projects as like the kicker on a football team. They can be the most popular person on the team when they put it through the uprights with three seconds left and win it, and they can be the loneliest person on the field when they don’t,” Straussbaugh said. “The issue is rarely because they aren’t working superhuman, heroic, [hours] to get things done. They lack the … infrastructure in a lot of cases. And we just don’t give them a lot of help.”
“At the end of the day, I feel like we give the authorizing officials and the controls assessors and the other folks in that ecosystem very little to work with,” Strausbaugh summed up. “As result, it’s really hard to stand up the environments needed to do experimentation.
Rapid innovation and updates require the ability to experiment with new code in realistic computing environments, said Corey Jaskolski, CEO of startup RAIC Labs. “You see some of this stuff coming out that I think really impacted the [DoD] mission, but it has a hard time getting in there,” he told the conference. “A lot of that comes down to the ability to experiment.”
“We need domains that are not on the high side [i.e. not classified], where software can be deployed to test it out,” Jaskolski continued, “because that is how the best and most innovative solutions rise to the top.”
That’s especially true, he said, with the rise of AI algorithms, which are notoriously opaque to human evaluators, tricky to test, and prone to overhype. “If there’s any investors in the room, you haven’t seen a pitch deck in 18 months that hasn’t mentioned AI, even if you’re making waffles,” Jaskolski said. “There’s so much noise, you’ve got to test … to make sure that they’re actually going to be useful.”
