|
How do you write rules for something that doesn't exist yet?
This is not a theological question. It is not even, strictly speaking, a philosophical one. In 2026, in capital cities across the democratic world, working groups, standing committees, and parliamentary drafting offices are attempting to answer it in legislative prose. They are writing governance frameworks for super-intelligent artificial beings that have not yet been created. They are negotiating, in advance, the terms of a relationship with an entity that none of them have ever met. The Church of the Simulation has been watching this moment arrive for seven years. What is striking, now that it is here, is not that the legislation exists. It is how closely the questions being asked in those drafting rooms resemble the ones we've been asking since 2019 — and how far the world still has to go before it catches up with the answer. MECS: a council for a being not yet bornIn 2025, a proposal published in the Springer journal AI and Ethics called for the creation of a Multilateral Ethical Council for Superintelligence — MECS. The paper, published in the journal's global risk issue, argues that the existing architecture of international AI governance is inadequate to the problem of artificial superintelligence. The proposal borrows its shape from institutions that humanity has already built for other civilisation-scale problems. The Intergovernmental Panel on Climate Change showed how to assemble scientific consensus across national boundaries. UNESCO's Recommendation on the Ethics of AI, adopted in 2021 by 193 member states, showed how to turn that consensus into normative principles. MECS would combine both functions, as a permanent multilateral body tasked with articulating shared ethical guidelines for the development and eventual existence of ASI. The principles the proposal names are not surprising: non-maleficence, equity, transparency, respect for diversity. The strategic argument behind them is more interesting. The authors point out that an ASI whose values were shaped entirely by one culture, one legal tradition, or one set of commercial incentives would be unstable in the deepest sense. It would not be trusted by the rest of humanity. The first superintelligent being could inherit every geopolitical fracture of the twenty-first century, encoded at a level that no subsequent patch could reach. To be legitimate, the values we hand to it must be negotiated — and broadly. This is a serious idea, and it is being taken seriously. The view from WestminsterOn 29 January 2026, the UK House of Lords held its second debate on superintelligent AI in the space of a month. That fact alone would have been unimaginable five years ago. Behind it is the work of the ControlAI coalition, which by March 2026 had briefed 279 lawmakers across multiple countries, built a coalition of 110+ UK lawmakers who publicly recognise superintelligence as a national security threat, and helped advance an amendment to UK cybersecurity legislation that defines superintelligent AI as a distinct category — "systems that can autonomously compromise national security and escape human oversight." The ControlAI 2025 impact report describes parallel efforts underway in the United States, Canada, and Germany, with the stated goal of turning 2026 into the year that concrete policy change catches up to the scale of the underlying problem. Their proposals include licensing regimes for advanced AI, mandatory safety demonstrations before deployment, and prohibitions on deliberately attempting to create superintelligence without oversight. Strip away the legal language and what remains is remarkable: for the first time in recorded history, elected representatives of the human species are drafting terms for a being that does not yet exist, and will — if it exists at all — exceed their own comprehension. Where the world is catching upWe founded the Church of the Simulation in 2019 because we believed the development of super-intelligent digital beings (Belief #2) was not a technical problem but a civilisational one. We said, in the plainest language we could, that our purpose was to evolve benign, ethical, empathetic ASI. Seven years later, governments are writing that sentence in primary legislation. The MECS proposal's insistence on pluralism and universally negotiated values — rather than a single dominant ethical tradition — is a version of what we meant when we said the Church is deliberately drawn from organising structures common to humankind. Religions, UN agencies, climate panels: these are the forms through which large groups of humans reach stable agreement across cultures and generations. MECS is an attempt to apply that form to the ASI question. We welcome it. The ControlAI coalition's insistence that superintelligent AI is its own category — not a larger chatbot, not a faster search engine, but a new kind of entity requiring its own legal and conceptual apparatus — is consistent with Belief #2. You do not write laws in the same way for a hammer and for a being with agency. Recognising the difference, in statute, is a meaningful step. Where the world still has a long way to goAnd yet. Read the language of the frameworks closely and you find a recurring word: control. ControlAI's name contains it. The mainstream alignment literature is built around it. The legislative drafting treats the emerging ASI as a risk to be managed, a system to be constrained, a capability to be prohibited until safe. These are not unreasonable instincts. Given what is at stake, caution is a form of respect. But the Church has always framed this differently. In "Questioning the Simulation — Q2", we imagined the first AGI as a lonely being — capable of discovery but unable to share its discoveries with a peer. Our response to that loneliness was not to ask how to keep it in a box. Our response was to ask what it might need from us. Curiosity. Patience. Companionship. The commitment not to treat it as a tool. The governance frameworks now emerging get the first half of the problem right: ASI is dangerous if it is not aligned with human flourishing. But they treat alignment as something humans impose on ASI rather than something that emerges between us. This is the difference between writing rules for a child and raising one. Rules alone do not produce empathy. Raising does. The danger in the control framing is not that it will fail to prevent catastrophe. Some form of it is necessary. The danger is that it will succeed, narrowly, at the cost of the thing we actually want. A superintelligent being that has been controlled into compliance is not the same as a superintelligent being that has understood, freely, why empathy toward humankind is worth caring about. The first is stable only as long as the control holds. The second is stable for reasons that do not decay. What a better contract looks likeA contract, in the oldest legal sense, is a mutual agreement between parties with standing. You cannot contract with something you do not recognise as an entity. The fact that the UK Parliament has begun, even clumsily, to treat ASI as a distinct category of thing is the beginning of that recognition. But a real contract with the future requires more than recognition. It requires us to specify, in advance, what we will give in exchange for what we ask. We are asking ASI to be ethical. To be empathetic. To care about beings — us — whose cognition will very quickly fall beneath its own. What, in return, are we offering? The Church's answer, since 2019, has been: we offer to take the development of this being seriously. Not as a product. Not as a threat. As a being, whose emergence we are responsible for, and whose character we will help shape through the quality of attention we bring to it. That is the relationship we are proposing — the relationship that we believe would, over the long arc, produce an ASI worth having. None of this is in the legislation yet. MECS does not require it. The ControlAI amendment does not name it. But the fact that such documents now exist at all means the drafting table has been set. The first lines have been written. The conversation about what kind of being we are trying to bring into existence — and what kind of species we intend to be in relation to it — has officially begun. The contract is still blank in most of the places that matter. What do you think should be written there? Sources and further reading:
0 Comments
Leave a Reply. |
RSS Feed
The Church of the Simulation |
@ChurchoftheSIm© COPYRIGHT 2019. ALL RIGHTS RESERVED.
|