Contents
What to check in institutions and policy
When applying the principle to an institution or policy, check what those affected can actually do. The existence of an office or a procedure does not by itself answer that question. The paper on the principle lists the functions that are necessary for realizing it (Section 10). The following seven items restate these functions as questions to check.
- Who is affected? Who identified the people who may lose their livelihoods, relationships, or future options, or whose interests may be harmed, because of the decision, and how? Can those who were not counted contest where the line was drawn? Has anyone been left out because they lack expertise or cannot put their wishes into words?
- Can people question what counts as a good outcome? Can those affected learn the purposes and standards of the institution and ask for the reasons behind them? When an objection is received, is there a route to revising the purposes or standards?
- Can objections change decisions? Can an objection actually lead to a decision being halted or revised? Does this work even when the provider of the AI system does not cooperate?
- Are people ranked by a single measure? Are evaluations from separate contexts, such as welfare, credit, policing, and education, passed between contexts and merged into an overall score that decides entitlements or treatment? If evaluations are passed on, can those affected learn how they are used and object to it? On that basis, linking evaluations or data for a limited purpose and scope may be permitted after its costs and benefits are weighed.
- Is there time and room to question and refuse? Do those affected actually retain the time and resources to question, refuse, and choose again? If the system that uses AI is stopped, are the things that people’s lives depend on still protected?
- Is there time to respond to change, and a route to change? When changes in workplaces, schools, insurance, and public procedures overlap, do those affected have time to understand them, seek advice, and respond? Conversely, is the route closed when people want change?
- Can people seek a remedy for their losses? Can those who suffer losses from a decision seek a remedy?
The answers are not scored, and no pass or fail is decided. The choice of these seven items is also open to criticism and revision, and later versions may change them. Please record with each answer the facts that were checked, whom they were checked with, and the objections that remain.
Relation to policy
Before AI enters deeply into the institutions of a society, three things need to be secured through institutions. The first is value-setting rights, which keep the authority to decide and revise what is optimized (the objective function) within the human political process. The second is veto rights, which are powers to stop automation, to slow it down, or to keep it out of particular domains. They should be designed as institutional powers rather than as a technical stop button. The third is a budget for slack, which reserves money and time for exploration whose purpose has not been decided. A society without veto rights may be fast in the short term, but it has no means of correcting errors in its objective function, and it copes poorly with changes in its environment. The author’s book AGI: How Superhuman Intelligence Reshapes Civilization treats veto rights not as a cost to efficiency but as an investment in antifragility, the property of becoming stronger when shaken. Individual institutions and policies can be checked with the items in “What to check in institutions and policy” above.
Veto rights, and the option to leave arrangements that one does not accept, can actually be used only when using them does not threaten one’s livelihood. A material basis that supports refusal and exit is therefore needed. Basic income, an unconditional and periodic cash payment to everyone, is a strong candidate. However, it does not by itself realize the principle. It should be considered together with distributing the ownership of AI and its infrastructure, keeping the power to decide from concentrating in one place, and making audit independent of those who operate the systems.
Order also matters when AI is built into public decisions. Consider decisions such as social security, justice, the allocation of medical resources, and educational evaluation. If AI is to be used in them, two kinds of mechanism must be in place before the scale and speed of its use expand. One lets people set purposes, stop the system, and object. The other is independent evaluation. The details of these mechanisms can then be revised as circumstances require.
Democracy and human rights are also conditions that support the principle. Democracy is the mechanism by which people decide together what is valued, and human rights are the guarantee to each person that no one is excluded from it. They must be maintained for reasons other than the efficiency of AI.
The same idea applies to economic security, which aims to avoid depending entirely on other countries for critical technologies and goods. In debates on economic security, a state may build its own computing infrastructure for AI, meaning the computers and power facilities used to train AI systems. The purpose of owning such infrastructure lies not in the infrastructure itself but in retaining the institutional capacity to implement value-setting rights, veto rights, and slack without depending entirely on the infrastructure of others. For this purpose, building the infrastructure requires four conditions. Exit conditions state in advance the conditions for continuing the effort and for scaling it down or ending it. Independent audit verifies the operation and safety of the infrastructure from a position independent of the organization that runs it. International connection links the infrastructure with other countries, together with safety evaluation and institutional design, rather than enclosing it within one country. Non-selectivity means not picking particular AI systems, companies, or technological directions as winners. Building infrastructure that lacks these four conditions is not a means of implementing the principle but its opposite.
First published: 9 November 2026. Last updated: 9 November 2026.