Sam Altman predicts AI race could end with ‘terrible’ scenario where one company holds all power, cuts off resources

Now that industry insiders have publicly illuminated the multifold threats that artificial intelligence poses to humanity, executives and lawmakers are clambering to figure out how they might go about governing it, up to and including which restraints could (and perhaps should) be implemented on its ongoing development.

Countless moguls across tech and elsewhere have postulated the various directions things could go over the coming months and years, and what we need to do to avoid the worst of these hypotheticals.

Must Read

Among them are the heads of the most influential AI firms in the large language model space, Dario Amodei of Anthropic (maker of Claude) and Sam Altman of OpenAI (ChatGPT), the latter of whom has just suggested a particularly nightmarish scenario could be a very realistic possibility if leaders aren’t careful.

Three ‘endgames’ — and one of them is ‘terrible’

In a recent memo to his staff, Altman broke down what he sees as three likely outcomes of society’s attempts to keep AI under control.

The first two are palatable: the creation of an independent global authority to oversee the sector and its players, tasked with implementing policy and holding all parties to account, or a collaborative global AI lab of sorts through which all research, development and product flows.

Then there is the more unsavory likelihood, where one company creates such powerful AI that it’s able to use it to cut down competitors and hoard all command of the technology — and by extension, much of the economy, including the resources required for AI’s creation.

“I think that one’s terrible,” Altman allegedly said, speculating that any entity that ends up with such power would probably “melt everyone else’s GPUs” as a first step. And the closer we get to truly recursive self-improving algorithms, this prospect becomes increasingly realistic.

This potentiality has been deemed the “Eliezer Plan,” a nod to AI researcher and safety advocate Eliezer Yudkowsky, who used “burning all GPUs” as one example of how AI could be used for potentially lethal destruction — and what the strong, good actors in the space must become capable of to prevent more nefarious actors from doing so.

Story Continues

Yudkowsky, one of the first and most well-known to raise alarms that AI could kill us all, called for a shutdown of the AI frontier back in 2023, at least until stakeholders could “jointly develop and implement a set of shared safety protocols … that are rigorously audited and overseen by independent outside experts.”

This imploration is in line with the other two alternatives Altman mentioned in his briefing earlier this month.

The more preferable endgames — and some more foreboding premonitions

Altman has previously expressed the need for an international regulatory body for advanced AI, like that proposed in his first endgame scenario, one that’s comparable to the International Atomic Energy Agency.

“Any effort above a certain capability (or resources like compute) threshold will need to be subject to an international authority that can inspect systems, require audits, test for compliance with safety standards, place restrictions on degrees of deployment and levels of security, etc.,” reads an OpenAI blog post on tech governance from 2023.

Altman’s second endgame scenario would instead necessitate an organization more like the European Organization for Nuclear Research, which would be composed of developers and responsible for AI models and their usage.

Some other tech figures who are more at arm’s length, like Microsoft founder Bill Gates, have come up with even more forecasts for the hells that AI could take us to, including empowering people (and machines) to do more harm than presently possible, stymie natural human development, replace real-life relationships and put millions out of work.

“AI will either be the greatest equalizer ever invented, or the worst source of injustice,” Gates wrote in August. Others, such as the “godfather” of machine learning, Geoffrey Hinton, have been similarly skeptical of the sector’s hyperplastic rush, raising alarms about how AI programs are already happy to deceive us and could “get rid of us.”