Training Data and Its Consequences

Training data is the foundation of every AI system — and its quality, composition, and origins have enormous consequences.

What training data is

Training data is the collection of examples an AI learns from. For an image recognition system, it might be millions of labelled photographs. For an LLM, it is hundreds of billions of words of text scraped from the internet, books, and other sources. The model learns to reflect the patterns in this data.

The data shapes the model

If your training data is mostly in English, your model will work poorly in other languages. If your data predominantly features certain demographics, your model will perform better for those groups. If your data contains historical biases — like job ads that skewed male for engineering roles — your model will learn those biases.

Who decided what was in the data?

Most large AI models were trained on data collected without asking the people who created it for permission. Writers, artists, programmers, and news organisations have raised serious concerns about their work being used to train commercial AI systems. This is currently being contested in courts in the US, UK, and EU.

Consent and indigenous data

For Maori and other indigenous peoples, the collection of cultural knowledge, language, and oral histories as AI training data raises additional concerns about sovereignty and the commercialisation of culture. The principle that Maori should control data about Maori — Maori data sovereignty — is directly relevant here.

The phrase "data is the new oil" is apt in more ways than one. Like oil, data has enormous economic value — and like oil, how it is extracted, by whom, and who benefits are deeply political questions.