AI Accountability and Explainability

As AI systems make more consequential decisions — about loans, jobs, bail, and healthcare — questions about accountability become urgent. Who is responsible when AI causes harm?

The accountability gap

When a human makes a harmful decision, we have mechanisms for accountability: they can be questioned, disciplined, or sued. When an AI system makes a harmful decision, it is often unclear who bears responsibility. The company that built the model? The organisation that deployed it? The data that trained it? The user who relied on it?

Explainability

Many modern AI systems — especially deep neural networks — are essentially black boxes. They produce outputs but cannot explain their reasoning in human-understandable terms. This creates a fundamental problem for accountability: if a person is denied a loan based on an AI decision, they are entitled to know why — but the system may not be able to say.

Existing frameworks

The EU AI Act, passed in 2024, establishes requirements for high-risk AI systems including transparency, human oversight, and the right to explanation. New Zealand is watching and likely to develop similar requirements. Australia released its AI Ethics Framework in 2019 and is developing more binding regulation.

What meaningful oversight looks like

Human oversight of AI does not mean a human rubber-stamps every decision — that would be neither scalable nor effective. It means: the human has enough information to meaningfully evaluate the AI output, sufficient time to do so, the authority to override, and incentives to do so when appropriate.

Accountability for AI systems is not a technical question — it is a political and institutional one. The choices societies make about who bears responsibility when AI causes harm will shape how AI is developed and deployed for decades.