From the kitchen: ChatGPT’s creators are surprised by its popularity

When it launched ChatGPT without much fanfare in late November 2022, artificial intelligence company OpenAI had few expectations. No one was prepared for the chatbot to become a mega-hit. Now the creators of ChatGPT are surprised by the way their creation has become a global “celebrity”.

The emergence of ChatGPT is viewed by OpenAI as a “preliminary research review,” said Sanhini Agarwal, who is in charge of policies at the company. It’s a “more polished” version of a two-year-old technology and, more importantly, an attempt to iron out some of its flaws by gathering feedback from the public.

“We didn’t want to present it as a big, fundamental advance,” says Liam Fedus, an OpenAI scientist who works on ChatGPT. It seems that the team was confused by the success of its “pre-release” but still took the opportunity to improve the technology, watching millions of people use it to fix the most tangible problems.

Since November, OpenAI has updated ChatGPT several times. The company even signed a multibillion-dollar deal with Microsoft and announced a partnership with Bain, a global management consulting firm that plans to use OpenAI’s generative AI models in marketing campaigns for its clients, including Coca-Cola. Outside of OpenAI, the buzz around ChatGPT is creating a “gold rush” about big language models.

Unexpected success

“It’s overwhelming. We were surprised and trying to catch up,” said Jan Leicke, leader of the OpenAI improvement team. “I checked Twitter often in the days after the launch and there was a crazy period where the feed was filled with screenshots of ChatGPT. I expected it to be intuitive to people and gain a following, but I didn’t expect it to reach this level of mainstream popularity,” John Shulman, co-founder of OpenAI, is candid.

“I think it was definitely a surprise to all of us how many people started using it. We work on these models so much that we forget how surprising they can sometimes be to the outside world,” admits Sandini Agarwal.

Part of the team’s bewilderment comes from the fact that most of the technology in ChatGPT is not new. ChatGPT is a fine-tuned version of GPT-3.5, a family of large language models that OpenAI released months before the chatbot. GPT-3.5 itself is an updated version of GPT-3 that appeared in 2020.

The company makes these models available on its website as application programming interfaces (APIs) that help other software developers incorporate the models into their own code. OpenAI also released an earlier, fine-tuned version of GPT-3.5, called InstructGPT, in January 2022. But none of these previous versions of the technology became such a hit.

Fine tuning

“The ChatGPT model is fine-tuned based on the same language model as InstructGPT, and we used a similar methodology to fine-tune it,” says Fedus. “We had added conversational data and tweaked the learning process a bit. So we didn’t want to exaggerate and present it as a big, fundamental advance. As it turned out, the conversation data had a big positive impact on ChatGPT.”

According to Shulman, the bot’s pure technical capabilities don’t really differ significantly between the models developed by the company—the difference is more that ChatGPT is more accessible and usable.

“In a way, we can think of ChatGPT as a version of an AI system that we’ve had for a long time. It is not a fundamentally more capable model than earlier versions. The same basic models were available almost a year before ChatGPT came out,” says Fedus.

What’s different is that now the bot is more in line with people’s expectations. “He talks to you in a dialogue, is easily accessible in a chat interface, tries to be helpful. I think that’s what people realize,” adds Fedus.

Human feedback

ChatGPT is trained in a very similar way to InstructGPT, using a technique called “reinforcement learning with human feedback” (RLHF). This is the secret ingredient of ChatGPT.

“We had a large group of people who read the ChatGPT prompts and answers and then said if one answer was preferable to another,” says Jan Leicke. “All of this data was then combined into a single training run. Almost everything is the same as what we did with InstructGPT’.

In an effort to make the bot more adequate, the developers try to make it useful and credible. An important element is the ability to conduct a dialogue.

“Sometimes the user’s request isn’t clear, so they have to ask follow-up questions,” says Leicke. “It should also be made clear that this is an AI system. It shouldn’t assume an identity it doesn’t have, it shouldn’t claim to have abilities it doesn’t have, and when a user asks it to do tasks it shouldn’t, it should say no.”

High bar

Because ChatGPT was built using the same techniques that OpenAI used before, the team believes it didn’t do anything different when preparing to release this model to the public. The researchers believe that the bar they set for previous models is high enough.

“GPT-3.5 already existed and we knew it was already safe enough,” says Agarwal. “You can’t wait until your system is perfect to release it. We’ve been testing beta versions for several months, and beta testers have had positive impressions of the product,” shares John Shulman.

“Our biggest concern was around factuality because the model likes to make things up. But InstructGPT and other big language models are already out there, and we thought that as long as ChatGPT is better than them in terms of fact-finding and other safety issues, it should be ready,” Shulman adds.

Battling Malice

OpenAI has been watching how people use ChatGPT since its launch. Developers watch how a large language model fares when placed in the hands of tens of millions of users who might want to test its limitations and find its flaws.

The team is taking the most problematic examples of what ChatGPT can produce—from songs about God’s love for priest-rapists to malicious code that steals credit card numbers—and using them to master future versions of the model.

“I definitely think that since ChatGPT became a hit, it’s helped crystallize a lot of problems that we knew existed—things that we want to solve as soon as possible. We know the model is still very biased. And yes, ChatGPT is very good at denying bad requests, but it’s also pretty easy to write prompts that make it not deny what we want it to deny,” says Agarwal.

“It’s been exciting to see the diverse and creative applications from users. But we are always focused on the areas we need to improve,” Fedus is frank. Multiple iterations allow available feedback to be used to refine the system. “As our technology evolves, new problems inevitably arise.”

The team has spent a lot of time looking at some of the most egregious examples people have found – the most malicious applications of the AI ​​system. They’re called “jailbreaks” – situations where someone manages to get the software to do something it shouldn’t. “Users have to try these complicated methods to get the model to say something bad. We are actively working on the issues right now,” shared Agarwal. All of these improvements will be built into the next iteration of the language model.

Meanwhile, in January Microsoft unveiled Bing Chat, a search chatbot that many believe is a version of OpenAI’s unannounced GPT-4. The use of chatbots by tech giants with multi-billion dollar reputations creates new challenges for the team tasked with building the popular language model.