The popularity of ChatGPT and the advance of generative AI technology have triggered a debate within the tech community about how dangerous the technology could become. Prominent AI leaders from OpenAI, Google and Microsoft warned this year that the tech could pose an existential danger to humankind, on par with pandemics or nuclear weapons. Other AI researchers have said the focus on those big, frightening risks allows companies to distract from the harmful effects the tech is already having. A growing group of AI business leaders say that the risks are overblown and that companies should charge ahead with developing the tech to help improve society — and make money doing it.
OpenAI has threaded a middle ground through this debate in its public posture. Chief executive Sam Altman said that there are serious longer-term risks inherent to the tech but that people should also focus on fixing existing problems. Regulation to try to prevent harmful impacts of AI shouldn’t make it harder for smaller companies to compete, Altman has said. At the same time, he has pushed the company to commercialize its technology and raised money to fund faster growth.
Madry, a veteran AI researcher who directs MIT’s Center for Deployable Machine Learning and co-leads the MIT AI Policy Forum, joined OpenAI this year. He was one of a small group of OpenAI leaders who quit when Altman was fired by the company’s board in November. Madry returned to the company when Altman was reinstated five days later. OpenAI, which is governed by a nonprofit board whose mission is to advance AI and make it helpful for all humans, is in the midst of selecting new board members after three of the four members who fired Altman stepped down as part of his return.
Despite the leadership “turbulence,” Madry said, he believes OpenAI’s board takes seriously the risks of AI. “I realized if I really want to shape how AI is impacting society, why not go to a company that is actually doing it?” he said.
The preparedness team is hiring national security experts from outside the AI world who can help OpenAI understand how to deal with big risks. It is beginning discussions with organizations, including the National Nuclear Security Administration, which oversees nuclear technology in the United States, to ensure the company can appropriately study the risks of AI, Madry said.
The team will monitor how and when OpenAI’s tech can instruct people to hack computers or build dangerous chemical, biological and nuclear weapons, beyond what people can find online through regular research. Madry is looking for people who “really think, ‘How can I mess with this set of rules? How can I be most ingenious in my evilness?’”
The company will also allow “qualified, independent third-parties” from outside OpenAI to test its technology, it said in a Monday blog post.
Madry said he didn’t agree with the debate between AI “doomers” who fear the tech has already attained the ability to outstrip human intelligence and “accelerationists” who want to remove all barriers to AI development.
“I really see this framing of acceleration and deceleration as extremely simplistic,” he said. “AI has a ton of upsides, but we also need to do the work to make sure the upsides are actually realized and the downsides aren’t.”
