ExplainerGovernmentNew
AI in government: the questions to ask about public-sector systems
When public agencies use AI to decide benefits, prioritize inspections, or answer the public, the key questions are what the system decides, who checks it, and how people can challenge it.
Evidence: 2 sources · 2 official records
The short version
- Governments use AI and automated systems for tasks such as processing applications, detecting fraud, translation, and answering questions.
- Errors in public-sector systems can affect people's benefits, legal status, or freedom, and those people often cannot opt out.
- Some governments publish registers of the algorithms their agencies use.
- Frameworks such as the OECD AI Principles and the NIST AI Risk Management Framework set out expectations for transparency and accountability.
AI in government is often less visible than AI in consumer products, but it can matter more to the people affected. A chatbot that gives a wrong answer about a product is an annoyance. A system that wrongly flags someone for benefit fraud can upend their life.
Five questions for any public-sector system
1. What does the system decide, or recommend? There is a large difference between a tool that drafts a letter and one that ranks people for investigation.
2. Who makes the final decision? "A human is in the loop" means little if the person has seconds per case and rarely overrules the system.
3. How was it tested, and on whom? Look for testing on the population it is used on, including checks for errors that fall more heavily on particular groups.
4. Can people find out and challenge it? Are people told when a system was involved? Can they appeal, and see the reasons?
5. Who built it, and under what contract? Many systems are bought from private vendors. Procurement contracts can limit what the public is allowed to know about how a system works.
Where to find information
Some governments publish algorithm registers or impact assessments. Freedom-of-information requests, audit office reports, court cases, and procurement records are also important sources. International frameworks, including the OECD AI Principles and the NIST AI Risk Management Framework, describe what good practice should look like.
How we cover government AI
We name the agency, the system, and the vendor where known, and we seek out the people affected as well as officials. An agency's description of its own system is reported as its description.
Limits and open questions
- Many governments do not publish complete lists of the AI systems their agencies use, so the full extent of use is hard to know.
Evidence and sources
- Document or data OECD AI Principles
- Document or data AI Risk Management Framework