Artificial Intelligence: Too Fragile to Fight?
Information Warfare Essay Contest—First Prize. Sponsored by Booz Allen Hamilton.
Automation—including AI—has persistent, critical vulnerabilities that must be thoroughly understood and adequately addressed if defense applications are to remain resilient and effective.
By Commander Edgar Jatho, U.S. Navy, and Joshua A. Kroll
February 2022
Proceedings
Vol. 148/2/1,428
You can become utterly dependent on a new glamorous technology, be it cyber-space, artificial intelligence. . . It’ll enable you. It’ll move you forward. But does it create a potential achilles heel? Often it does.1
—Admiral James Stavridis
Artificial intelligence (AI) has become the technical focal point for advancing naval and Department of Defense (DoD) capabilities. Secretary of the Navy Carlos Del Toro listed AI first among his priorities for innovating U.S. naval forces. Chief of Naval Operations Admiral Michael Gilday listed it as his top priority during his Senate confirmation hearing.2 This focus is appropriate: AI offers many promising breakthroughs in battlefield capability and agility in decision-making.
Yet, the proposed advances come with substantial risk: automation—including AI—has persistent, critical vulnerabilities that must be thoroughly understood and adequately addressed if defense applications are to remain resilient and effective. Current state-of-the-art AI systems undergirding these advances are surprisingly fragile—that is, easily deceived or broken or prone to mistakes in high-pressure use.
Machine learning (ML) and especially modern “deep learning” methods—the very methods driving the advances that make AI an important focus area today—are distinctly vulnerable to deception and perturbation.3 Often, human-machine teaming is thought to be the solution to these issues, but such teaming itself is fraught and unexpectedly fragile in persistently problematic and counterintuitive ways.4
https://www.usni.org/magazines/proceedings/2022/february/artificial-intelligence-too-fragile-fight
