Bureaucratic bastardry: robodebt/debt recovery, AI and the stigmatisation of citizens by machines and systems

Abstract

Automation in public administration is inevitable and can bring great benefits. Seventy years ago, Isaac Asimov foresaw the need to protect humans from their creations in his novel I, Robot, drafting three laws to ensure robots: 1) did not injure a person, or allow them to be harmed; 2) obeyed orders, that did not conflict with the first law; 3) protected their own existence without compromising the first or second laws. The first two laws can apply to the automated systems created by government to administer public service. This paper examines the failed policy to automate welfare debt collection in Australia. Despite repeated warnings the malign policy caused considerable harm to clients and resulted in a billion-dollar settlement against the government. Using a framework centred on the concepts of organisational evil and bureaucratic animosity, we suggest that such complex undertakings could benefit from reference to Asimov's laws.