Evidence or agenda? Policy analysis in the age of AI

Policy Analysis

How will policy analysis change in a world of AI? This paper from RAND paints a number of scenarios.

Key takeaways

• Policy analysis is at a turning point because AI, falling trust in expertise and power spreading away from government are weakening the institutions that once gave analysis its place in public life.
• The field was built for the post-war world, when authority sat with a few public bodies, information was scarce, and expertise was rarely challenged.
• Policy problems are now more interconnected and prone to sudden shifts, so analysts need methods that test strategies against many possible futures rather than relying on a single forecast.
• The authors use two uncertainties, how much people trust expertise and how far AI replaces humans in doing analysis, to sketch four futures for 2035 (see image).
• As AI drives the cost of gathering information towards zero, an analyst’s value shifts from producing information to judgment: framing the right questions, spotting blind spots and making value choices explicit.
• The model they favour is “centaur teams,” where AI extends human judgment rather than replacing it, so analysis gets faster without losing critical scrutiny.
• In contested information environments, the key line is between conviction grounded in evidence and agenda-driven distortion, so “boundary-spanning ethics” should be taught as a skill.
• Organisations will compete less on owning information and more on validating AI outputs, handling deep uncertainty, co-producing analysis with decision-makers and building narrative into the analysis from the start.
• The field’s legitimacy will depend less on its internal standards and more on showing improvements that the public recognises, which means being open about how it works and linking analysis to implementation and results.
• Small updates to tools and methods won’t be enough, and universities, research organisations, professional associations and funders need to act together on the field’s first principles: define problems, compare alternatives and reason transparently under uncertainty.

Comment

This is a sobering paper for believers in high-quality policy analysis. Policymakers across the political spectrum value that analysis less and less as 'agenda-driven distortions' increasingly override a search for truth.

Will AI simply tell us what we want to hear, creating competing narratives people think are true, amplifying the political polarisation we are already seeing as people retreat into their own political bubbles, or will we find ways to reverse these trends?

What next?

Do you have a well-thought-out approach for using AI in policy analysis?

---

• Participation builds trust, not framing: insights from a national energy-transition experiment - A research paper looking at the unintended consequences of framing.
• Top Ten Behavioral Biases in Project Management: An Overview - Although focused on project management, this research paper also applies to policy reforms.
• Peeling The Banana - A blog on diagnosing and solving problems in government.

---

• New: I have added a resources page that is available to everyone now (you don't need to sign in). If you know of good resources I should include, please let me know.
• Coming soon: The toolkit is nearly ready to go live. Watch this space!

---

Tell me what you think

Email me at russell@policyreformtoolkit.com with your thoughts, questions or requests. I read every message, and your feedback shapes what the Toolkit does next.

Russell