In a meeting on November 16, 2024, U.S. President Joe Biden and Chinese President Xi Jinping found common ground on artificial intelligence (AI), recognizing the need for human beings to have ultimate authority over decisions to use nuclear weapons. Their agreement represents a rare moment of cooperation between two countries that are constantly competing in both AI and nuclear innovation. The rapid advancement in the capabilities of artificial intelligence and its increasing accessibility to the public is becoming a defining theme of the 21st century. The development of hyper-advanced machine learning algorithms, like those used in generative AI models like GPT 4, continues to increase the autonomy of artificial intelligence. As computers become less reliant on human intelligence, humans become more reliant on computers. Meanwhile, advancements in nuclear technology and renewed nuclear proliferation now tempt state actors to leverage the machine learning power of artificial intelligence and automate the decision to use a nuclear weapon. Given these incentives and the current capabilities of AI, the decision world leaders are faced with is not if AI should be used in nuclear command and control but to what extent.
In order to properly assess the risks of relying on the decision-making of artificial intelligence when it comes to nuclear weapons use, an understanding of existing nuclear weapons systems and the factors that would incentivize states to automate nuclear command and control protocols is necessary. The U.S. Nuclear Command, Control, and Communications (NC3) system serves five main functions for the U.S. nuclear weapons program:
- detection, warning, and attack characterization
- nuclear planning
- decisionmaking conferencing
- reception of presidential orders
- management and direction of forces
These primary functions are executed according to a rigid chain of command system involving a number of different government and military agencies. Ultimately, the President has the sole authority to authorize the use of a nuclear weapon.
Since the Cold War, the U.S. and other nuclear weapons states have used artificial intelligence to improve efficiency in various sectors of the NC3 system. For example, the U.S. uses artificial intelligence automation in its early-warning systems to detect an incoming nuclear attack and guide outgoing attack missiles. These types of artificial intelligence tools generally belong to a category known as “rule-based AI,” which refers to AI models that operate based on direct programming from a human source. Currently, rule-based AI in nuclear weapons procedures is used in conjunction with human decision-making.
Artificial intelligence applications in nuclear command and control are contingent upon the principle that nuclear weapons states would only detonate a nuclear weapon in response to a foreign nuclear attack. Currently, the only nuclear weapons state with a comprehensive “no first use” policy is China. India also has a conditional “no first use” policy that warrants a nuclear response to chemical or biological weapons attacks. Despite attempts to pass legislation declaring a “no first use” policy in the U.S., current law allows for the U.S. to preemptively strike an adversary with nuclear weapons. However, former Defense Secretary William Perry acknowledges that no “rational president” would ever authorize the first use of nuclear weapons.
Because nuclear weapons states have demonstrated an adherence to the doctrine of “no first use”, the need for timely response to incoming nuclear attacks is critical. With the current state of nuclear weapons technology, governments would likely have only minutes to detect and respond to nuclear threats before they are attacked. However, the decision on how to proceed once an attack has been detected has become more and more difficult due to the proliferation of nuclear weapons. During the Cold War, the U.S. military was able to develop detailed plans for responses to nuclear attacks because their only threat was the Soviet Union. Today, there are nine countries that possess viable nuclear weapons. Additionally, intelligence on adversarial nuclear powers has also become more ambiguous. For example, North Korea has been known to conceal its nuclear weapons program from international inspection and intelligence. Growing uncertainty regarding nuclear threats and the need for timely response to an attack has forced nuclear weapons states to look for ways to increase the efficiency of their nuclear response capabilities.
Existing difficulties in nuclear command and control operations cast additional doubts on the potential for safe integration of artificial intelligence. In 2015, The Union of Concerned Scientists published a report detailing historical examples of errors in command and control that nearly resulted in incidental or ill-advised nuclear detonation.
The first category of errors identified in the report related to ballistic missiles are instances of ambiguous data generated by early warning sensors. An example of this type of error was in 1995 when Russian early warning systems detected a Norwegian scientific rocket and classified it as a nuclear threat, nearly leading to a retaliatory strike from Russia. Even though Russia was previously notified about the launch, there was a lapse in communication.
Another category of errors identified in the report is false alarms caused by natural phenomena. In 1983, a Soviet early warning satellite detected five land-based missiles launched by the U.S. at the Soviet Union, which turned out to be reflections of the sun on the clouds that misled the missile detection system. According to the report:
“Since the satellite was found to be operating properly, following procedures would have led him [the Soviet officer on duty] to report an incoming attack. Going partly on gut instinct and believing the U.S. was unlikely to fire only five missiles, he told his commanders that it was a false alarm before he knew that to be true.”
In this case, a nuclear strike was saved by the “gut instinct” of the Soviet officer. Clearly, there are many challenges to the nuclear command and control framework exemplified by both human error and machine error.
While nuclear weapons states demonstrate a growing interest in leveraging the power of AI to improve the efficiency and accuracy of nuclear response, AI developers are showing more of an interest in military applications for their products. As mentioned, basic forms of artificial intelligence such as Rule-based AI have been used in nuclear command and control procedures since the Cold War era. Rule-based AI mimics human decision-making by operating based on conditions directly programmed by humans. This kind of artificial intelligence uses a network of syllogistic “if-then” instructions to make decisions based on input data. Because of the simplicity of the programmed instructions and the clarity in the decision-making process of the model, rule-based AI typically poses minimal risk of error. However, as the landscape of nuclear threats widens, leaders are looking at more advanced AI models that are able to process larger amounts of data more efficiently.
Recent breakthroughs in artificial intelligence have been centered around advancements in machine learning. Machine learning models differ from rule-based models in the source of their programming. Machine learning can be generally defined as “the capability of a machine to exhibit intelligent human behavior” – integral to that is the model’s ability to teach itself in the absence of human programming. Instead of operating based on a stagnant knowledge base of instructions, a machine learning model is able to constantly update its own instructions based on new data it receives.
There is no doubt that recent advancements in machine learning have made the world a safer place in many respects. Thanks to modern machine learning, self-driving cars can keep passengers safer than any human driver possibly could, banks can more effectively detect credit card fraud, and medical imaging can be more accurately analyzed to diagnose illnesses. Despite the many positive impacts modern machine learning AI has had on human lives, there are still several technical shortcomings.
The first technical issue that is characteristic of modern AI models is human bias. While the algorithms used by modern machine learning models have innate capabilities to self-modify, they are originally programmed by human beings. Therefore, the algorithms themselves can be subject to the same bias as the humans who created them. While this is also true of rule-based AI models, machine learning algorithms have the ability to extrapolate human bias in their programming. In generative AI models that use machine learning algorithms, such as GPT 4, human bias could be applied in unintended situations that prove dangerous. There are countless examples of human bias impacting AI outputs, such as the model Amazon used to filter resumes, which had a predetermined bias toward male applicants.
A 2022 report by Arms Control Today outlines the risks of human bias on AI as it relates to nuclear weapons. Modern machine learning algorithms are not developed by people with knowledge about nuclear threat detection, meaning the AI developer’s lack of understanding about nuclear command and control might create a human bias in their model. Although an AI model might proceed flawlessly according to its instructions, bias in the instructions themselves could lead to unpredictable outputs.
Another problem with modern AI models pertains to outputs generated from inaccurate information. A concerning characteristic of modern AI models is that their decision-making is solely based on the data they receive. This means that there is no way for an AI program to determine whether the data it receives is accurate or best suited for the job it is designed to perform. Imagine an AI model designed to read radar data and detect incoming missile attacks. If the model has been trained, even if inadvertently, using data from inaccurate radar readings, then it may produce an output based on the faulty data.
Even if an AI model is trained with accurate and relevant data, its algorithm still has the ability to produce unpredictable outputs that are unsupported by the training data. Applied to incoming missile detection, an AI model could be trained with completely accurate radar data and still produce an unintended output that may falsely identify or fail to detect a threat. AI outputs that do not align with the model’s provided training data are known as “hallucinations.”
The increasing complexity of modern machine learning algorithms come with increasing ambiguity over the steps AI models take to produce an output. The clarity of an AI model’s decision-making process is referred to by experts as “explainability.” In a highly explainable AI model, it is easy to discern each step the model takes from receiving data to generating a response. Models that use multi-layered machine learning algorithms are less transparent than those that use very simplistic algorithms, like rule-based AI models. Making decisions based on unexplainable AI-generated outputs would strip humans of the ability to check for errors in reasoning and refine models for future use.
Finally, unlike human intelligence, AI is vulnerable to external cybersecurity threats and attacks. The National Institute of Standards and Technology issued a report in January outlining the various methods and means used by adversaries to disrupt or influence complex machine learning models. The report breaks down cybersecurity attacks targeting AI models into four categories: evasion, poisoning, privacy, and abuse. These categories of cyber attacks involve adversaries gaining access to or manipulating AI training materials and input data with the goal of finding weaknesses in the model or generating unintended outputs. These cybersecurity threats call into question the extent to which AI should be implemented in nuclear command and control systems.
Artificial intelligence is very good at presenting data to human beings in a way that is manageable. However, when it comes to decision-making, AI is flawed. Artificial intelligence applications in nuclear command and control networks should be focused on assisting in areas where humans have demonstrated errors in decision-making. Reliable and explainable artificial intelligence models should be used in conjunction with human analysts for nuclear detection, warning, and attack characterization. However, the ultimate decision to deploy or detonate a nuclear weapon should only ever be made by human beings.