Towards a principle for the human supervisory control of robot weapons

Abstract

The aim is to explore the delicate balance between human reasoning and the legal requirements for the appropriate supervisory control of robot weapons. We start by examining the limitations of automatic versus aided target recognition and then review some of the relevant psychological literature on reasoning. A new framework is introduced that reframes autonomy/semi-autonomy in terms of levels of supervisory control. This allows for greater transparency in command and control and the allocation of responsibility. Finally, so-called, human supervised autonomy is assessed in terms of the supervisory control framework. Keywords: autonomous weapons, supervisory control, robot weapons, international humanitarian law, reasoning In a world where computing is taking us to new levels of automation, we must ensure that the decision to kill remains firmly under human control. Most new technological artefacts are controlled by computer chips and the technologies of violence are no exception: computer devices are becoming ubiquitous for most modern weapons and guidance control systems. Currently almost all of these weapons are under “supervisory control”, where a computer program mediates human control.1 Humans need to exercise meaningful control over weapons systems to counter many of the problems that arise from automation. The United States Department of Defense points out a number of the potential difficulties with entirely computerising robot weapon: human error, human-machine interaction failures, malfunctions, communications degradation, software coding errors, enemy cyber attacks, infiltration into the industrial supply chain, * Thank you to Maya Brehm, Article 36 and Dr. Amanda Sharkey, Computer Science, University of Sheffield for helpful comments on earlier drafts of this article. 1 Supervisory control stands in contrast to direct human control such as aiming a conventional rifle and pressing the trigger manually. 2 «The exercise of control over the use of weapons, and, concomitant responsibility and accountability for consequences are fundamental to the governance of the use of force and to the protection of the human person». Article 36 (2013) Memorandum for delegates to the Convention on Certain Conventional Weapons (CCW). Downloaded from: http://bitly.com/NJoVG3. Last accessed March 2 2014. To appear in POLITICA & SOCIETA 2014 2 jamming, spoofing, decoys, other enemy countermeasures or actions, unanticipated situations on the battlefield. 3 Even though such difficulties are well known, there is ever-increasing push by states to develop autonomous robot weapons that could move outside the reach of human supervisory control. The US has conducted advanced testing on a number of autonomous weapons platforms such as X-47b – a fast subsonic autonomous jet that can now take off and land on aircraft carriers, the Crusher – a 7 ton autonomous ground robot, and an autonomous hunting submarine. The Chinese are working on the Anjain supersonic autonomous air-to-air combat vehicle. The Russians are developing an autonomous Skat jet fighter. Israel has the autonomous Guardium ground robot and the UK is in advanced testing of the Taranis – a fully autonomous intercontinental combat aircraft. Currently, public reports on the testing of these robot devices have only been about the weapons-carrying platforms and not the weapons systems as a whole. Although autonomous robot platforms can have multiple useful purposes, arming them to select targets and attack them without deliberative human supervision raises serious legal, ethical, technical and international security concerns. The main concern of this paper is to explain the psychological underpinnings and problems with human supervisory control and to emphasise the need for a comprehensive method to ensure that appropriate deliberative human reasoning is utilised. 1. Automatic versus aided target recognition A major problem with weapons systems in which a computer program selects targets a