In the Superintelligence Race, More Information Can Mean More Danger
A simple model of an AI race finds that teams are safest when they know nothing about each other's capabilities. Openness turns out to have several dimensions, and IS knows something about designing disclosure.
Much of what I read in IS treats information as something that helps. Better information leads to better decisions, and transparency is a value we rarely question. A model of an AI race from 2013 argues that in this setting more information can make things worse. I have not been able to stop thinking about it.
Armstrong, Bostrom and Shulman (2013) present "a simplified model for analysing technology races". Their starting assumption is that "the first AI will be very powerful and transformative", so "each team is incentivised to finish first", "by skimping on safety precautions if need be." The teams in their model decide how much safety to give up for speed. The result that surprised me is in a section they call the curse of too much information. "Surprisingly, information also increases the risks". In their words, "the more teams know about each others' capabilities (and about their own), the more the danger increases." They state the extreme case plainly. "The no-information case is always safer than the other two cases". And "It is always better if none of the teams have any idea about anyone's capability."
They do not offer secrecy as a simple fix. "Such secrecy can interfere with trust building, though, making it hard to reach agreements between teams if such agreement is needed." They see more promise in having fewer teams. "In terms of leverage, the most helpful intervention we could undertake to reduce the number of teams is to encourage groups to merge."
Bostrom (2017) takes the question further and breaks openness into parts. "Openness is thus not a binary variable, but a vector with multiple dimensions that each admits of degrees." Some parts look good. "Some forms of openness are plausibly positive on both counts (openness about safety measures, openness about goals)." Others are risky. Openness about source code, science and possibly capability "could lead to a tightening of the competitive situation around the time of the introduction of advanced AI, increasing the probability that winning the AI race is incompatible with using any safety method that incurs a delay or limits performance". He also names the kind of openness he finds desirable. "Partial openness that enables outsiders to contribute to an AI project's safety work and to supervise organizational plans and goals appears desirable."
The International AI Safety Report (2025) describes the information problem from the side of governments. "There is an information gap between what AI companies know about their AI systems and what governments and non-industry researchers know." So two things seem to be true at once. Outsiders know too little to supervise, and competitors who know too much about each other may race harder. Bostrom's vector is the way out of that apparent contradiction. Openness about safety measures and goals helps outsiders supervise. Openness about capability helps competitors race.
This is where I think IS can contribute something concrete. We study information disclosure between firms, to regulators and to the public. I think what is disclosed, to whom and in what form matters as much as how much is disclosed. A disclosure regime for frontier AI could be designed along Bostrom's dimensions, with safety practices and goals made public, and capability details shared with supervisors but not with rivals. Whether that works is an empirical question, and it is a question about the design of information systems between organizations.
There is also a link to the note on organizational risk. Hendrycks and colleagues worry about safetywashing, which is an organization saying more about its safety than is true. A disclosure regime that asks for safety practices would have to deal with that, because a public statement about safety measures is only useful if someone can check it. The same problem shows up in the alignment literature as a gap between what an organization promises and what its AI does, which I take up in a note on alignment as an organizational promise.
The limits here are large. Armstrong and colleagues built a simplified model on purpose, and Bostrom's paper is an argument, not a test. Bostrom writes that "In a tight competitive situation, it could be impossible for a leading AI developer to slow down or pause without abandoning its lead to a competitor." He also writes that "The larger the pool of competitors, the harder it would be for them to all coordinate to avoid a risk race to the bottom." Both claims are plausible, and as far as I can tell from these sources neither has been measured in real firms. That is not a reason to ignore them. It is a reason to study them, and the study would be about information, incentives and organizations, which is home ground for IS.