Who Answers When Ai Gets It Wrong?

Eba Blog
Who answers when AI gets it wrong?

Who answers when AI gets it wrong? Human responsibility under South African law
2026-10-13

Who answers when AI gets it wrong? Human responsibility under South African law

When an AI system makes a costly mistake, the system does not appear in court. The people who chose, configured and supervised it do.

An AI tool can approve a transaction, reject a customer or produce a legal citation. It cannot owe a fiduciary duty, face a regulator or answer questions under cross-examination. Behind every automated decision sits a chain of human choices: what to buy, what to train, what to test, what to monitor and when to stop.

The law already assigns responsibility

South African law does not need to treat a machine as a legal person. Contract, delict, company law, consumer protection, data protection and professional negligence already ask who owed the duty, who authorised the conduct, who could foresee the harm and who failed to act reasonably.

POPIA places responsibility on the responsible party. Section 71 regulates certain decisions based solely on automated processing when those decisions have legal or substantial effects. Even where an exception applies, safeguards are required. "The model decided" is not a defence. It shows that an organisation built a decision process without anyone in charge.

King IV points the same way. Technology and information governance belong inside corporate governance. Directors can delegate implementation. They cannot delegate the board's duty to oversee risk, performance and ethical conduct.

The liability chain can include the developer, provider, deployer, professional adviser, executive committee and board. Indemnities move financial exposure between these parties, but they do not remove negligence. A "human in the loop" with no authority and 600 alerts to clear is not meaningful oversight.

Two South African examples

In Mavundla v MEC: Department of Co-Operative Government and Traditional Affairs KwaZulu-Natal, the Pietermaritzburg High Court dealt with a filing in which seven of the nine cited cases did not exist. The court criticised the conduct, made a personal costs order for additional appearances and referred the matter to the Legal Practice Council. Cliffe Dekker Hofmeyr's analysis records that senior professionals stay responsible for verifying authorities and supervising junior work. Verification has to happen before filing, not after a judge finds the problem.

In April 2026, government withdrew the draft National AI Policy after fictitious sources were found in its reference list. The official announcement said the most plausible explanation was unverified AI-generated citations and called the failure an integrity and credibility problem. The minister questioned quality assurance, promised consequence management and described vigilant human oversight as essential.

Regulators abroad are moving the same way

Article 14 of the EU AI Act requires high-risk systems to be designed for effective oversight by natural persons. Those people must understand the system's limitations, spot anomalies, resist automation bias, interpret outputs, override decisions and stop the system when necessary.

The US NIST AI Risk Management Framework is voluntary. Its four functions are Govern, Map, Measure and Manage, and they give organisations a practical structure for managing risk across a system's life.

In the United Kingdom, the proposed Public Authority Algorithmic and Automated Decision-Making Systems Bill focuses on transparency, impact assessments, monitoring and appeal rights for automated decisions in the public sector. The OECD AI Principles list accountability next to transparency, robustness and human-centred values.

The Stanford 2025 AI Index recorded 233 reported AI incidents in 2024, an increase of 56.4% on 2023. Responsible-AI evaluations remained uncommon.

Who pays when AI makes an expensive mistake

Several parties might. The provider could face product, contractual or misrepresentation claims. The deployer could be liable for negligent selection, integration or monitoring. Directors and executives could face governance consequences. Professionals could face disciplinary action. Insurers might see exclusion clauses tested.

Causation will be the contested issue. The loss might trace to defective training data, model drift, a configuration choice, an integration failure, a misleading interface, an ignored alert or a human override. Sophisticated systems spread decisions across layers, which means more documents, experts and cost to establish who was responsible.

Contracts matter, but they do not replace governance. Your agreements should state who warrants data quality, system performance, lawful use and regulatory compliance. They should also say who carries audit and notification duties, who keeps the logs and who controls model updates.

Five steps for your business

First, name an accountable executive for every high-impact system. Give that person authority, budget and an escalation route. A committee advises, but one named person answers.

Second, classify decisions by consequence. Decisions about credit, employment, litigation, healthcare and pricing need more testing, independent review and human sign-off than an automated diary reminder.

Third, make human review real. Reviewers need time, context, competence and the power to disagree with the output.

Fourth, keep evidence. Log model versions, inputs, outputs, confidence scores, overrides, alerts and approvals. Set retention rules that anticipate litigation, regulatory investigation and insurance claims.

Fifth, rehearse failure. Run exercises with legal, compliance, technical, operational and communications teams. Decide in advance who stops the system, who notifies affected people and regulators, who preserves evidence and who speaks publicly.

Start with one task this week. List every automated decision in your business that could cause material harm. Name the human owner of each one. Then test whether that person can understand the decision, challenge it and stop it. Where the answer is no, fix that gap first.

Adapted from "Guarding the gate: Human responsibility in algorithmic liability"

by Alicia Koch, published on GoLegal on 6 October 2026 (provided by AJS).

Recent posts