Australian Government Warned Over AI and Automated Decision-Making Risks

The federal government is facing mounting pressure over its use of algorithms and artificial intelligence, with independent watchdogs warning that a lack of oversight could lead to another major administrative scandal. Dr. Catherine Williams, chief executive of the independent think tank the Centre for Public Integrity, cautioned on Thursday that authorities are sleepwalking into a scenario reminiscent of the controversial robodebt program.

The warnings accompanied the release of the centre's new report titled “Computer Says No,” which examines automated decision-making across government operations and advocates for immediate legislative reform. Among its eight primary recommendations, the report calls for mandatory audits, a public register for all automated systems, and the establishment of an independent oversight body with powers to investigate government agencies.

The think tank's concerns were largely catalyzed by the government's integrated assessment tool, an algorithm utilized to determine the level of home support older citizens are entitled to receive. Investigations have shown that the tool frequently under-assesses individual needs, while human assessors have reportedly been barred from overriding flawed algorithmic decisions.

The robodebt scandal, which prompted a royal commission after the Labor Party came to power in 2022, involved an automated process that averaged Centrelink recipients' fluctuating incomes to calculate assumed debts. The royal commission ultimately found that the modeling routinely miscalculated liabilities, resulting in many beneficiaries being falsely accused of owing money in a process that severely impacted numerous lives.

In response to the growing concerns, the attorney general's department has been working on an automated decision-making framework, with public consultations concluding in January 2025. Attorney General Michelle Rowland defended the government's stance, stating that officials are dedicated to ensuring the safe and responsible use of technology to deliver timely services while implementing recommendations from the robodebt royal commission.

Despite these assurances, critics point out that previous recommendations have already been sidelined. A commonwealth ombudsman report published in October revealed that transparency recommendations stemming from the royal commission were ignored during the development and rollout of the aged care assessment tool. Rowland did not specify when a draft framework would be made public or whether it would apply retrospectively.

Independent MP Kate Chaney has also stepped up pressure on the administration, having introduced a private member's bill in September to establish the nation's first legislated, mandatory framework for automated decision-making. Chaney argued that government departments currently operate with inconsistent standards and poor transparency as they seek cost savings through automation, noting that rebuilding public trust requires clear review pathways and accountability.