Inflection CEO Mustafa Suleyman has already started work on his next model, aiming to surpass the capabilities of GPT-4.
Inflection AI
Inflection AI, the startup behind conversational chatbot Pi, has unveiled a new AI model that the company says can outperform two popular alternatives developed by Google and Meta – and closely follows OpenAI’s larger flagship model, GPT -4.
Called Inflection-2, the model performed better than Google’s PaLM Large 2 model previously announced in May on a number of standard tests, Inflection said, while beating Meta’s widely developed open source LLaMA 2 model on different measurements. Overall, Inflection’s model is the best performing of its size, the startup said. It is behind only GPT-4, OpenAI’s flagship model, which is considered much larger.
“We believe that we are only at the very beginning of this evolution curve and that the new capabilities that are going to emerge are truly breathtaking,” CEO Mustafa Suleyman said in an interview.
The newly released model will soon be integrated into Pi, the chatbot launched by Inflection in May. First, he needs a little more extra work called “alignment” to teach him Pi’s tone and response style, and to help Pi function better while absorbing up-to-date information without additional hallucinations, said Suleyman.
“Whether you want to have a sensitive conversation about race, gender, politics, OpenAI, or any other controversial topic of the day, Pi very subtly and carefully engages with you in a factual way, with real-time access to the information. ” he said. Pi will be updated with the new model “very soon, it won’t be long,” Suleyman added, while refusing to provide a date.
He wouldn’t provide updated user numbers on Pi, but said “it’s extremely popular, with huge retention.” Two weeks ago, OpenAI revealed its free ChatGPT service had reached 100 million weekly users, just before the board temporarily fired CEO Sam Altman on Friday.
It’s hard not to consider the big version of Inflection’s language model, billed as “the second most successful LLM in the world today,” in the context of the turmoil affecting the industry’s best-known model maker . Earlier Wednesday morning, OpenAI announced that Altman was returning as CEO of OpenAI, after a weekend of negotiations among allies for Altman’s return and OpenAI employees signing a petition calling for mass resignation of their board of directors.
At Inflection, Suleyman insisted that his startup, which raised $1.3 billion in funding earlier this year, had not moved up the release date for Inflection’s new model, despite public remarks previous ones which seemed to suggest a release at the end of the year. This outing had actually been expected for a week, he said, with the training having finished last week.
To form Inflection-2, the startup used 5,000 Nvidia H100 graphics processing units, or GPUs, compared to several thousand older A100s that formed its predecessor. The new model, which Suleyman said was faster and cheaper to train, still handled a massive number of operations (10^25 FLOPs, or floating point operations). Inflection partners closely with Microsoft, Nvidia and CoreWeave to manage its computing cluster.
Inflection tested its new model against a popular benchmark of high school and professional level tasks known as MMLU, where models must answer questions on 57 topics ranging from knowing the world to problem solving and ethics.
Inflection-2 performance compared to other models.
Inflection AI
Suleyman said Inflection-2 outperformed the larger 70 billion parameter version of LLaMA 2, Grok-1 from Elon Musk’s xAI startup, Google’s PaLM 2 Large and Claude 2 from startup Anthropic, trailing only GPT-4. The new model beat the LLaMA 2 and PaLM 2 models on all but two scientific response criteria, Inflection reported; it also performed best on two of the three question-and-answer tests, losing in one to PaLM 2 Large. On four math and coding tests, it performed respectably, even though those areas aren’t explicit goals, the company added; it does, however, lag significantly behind GPT-4 out of the two for which OpenAI has shared results.
Although such criteria may not seem important to anyone except an AI researcher or developer, small improvements can make the difference between a “hacked prototype” and a “production-grade, reliable, high-quality” model. Suleyman said. Overall, Suleyman said Inflection-2 could be considered best-in-class for its size – “very, very close” to GPT-4. Starting today, Inflection is focusing on training toward its next model, which it says will grow to 100 times the size of its predecessor within a year and reach OpenAI’s market standard.
Speaking hours before OpenAI’s messy board battle reached a tentative resolution, Suleyman urged the public to approach those involved “with empathy and forgiveness.” “There are a lot of very well-meaning people there,” he noted, taking particular shout out to co-founder Ilya Sutskever, originally one of the directors who fired Altman, and who later announced that he regretted this decision on Sunday evening. (It was not immediately clear whether Sutskever would stay at OpenAI, but he liked posts from the company and Altman announcing the return to X.) Sutskever worked for Suleyman as a contractor and consultant at DeepMind, the AI startup that he sold to Google in 2011. “I have great respect for him, not only technically, but he has a lot of principles and is very sincere. I guess he and the rest of the team have genuinely good intentions,” Suleyman said.
Inflection and Suleyman have already been franc on AI security, signature President Biden’s voluntary order in July, among other measures. He also pledged not to let Pi answer any election-related questions or participate in creating materials for political campaigns. Suleyman recently published a book that details some of the risks of AI, called “The Coming Wave.”
“Obviously there were some very, very significant mistakes made,” he said of the OpenAI board and the actions of its unusual nonprofit board, which had been slandered by some in the tech industry in recent days. “I hope people will try to forgive and see this as a minor failure in the overall journey of building better companies that do the right thing and try to make the world a better place with new governance structures.” »
That said, Suleyman is also a capitalist: when asked if OpenAI’s stumbles (ChatGPT was down for at least part of Monday and Tuesday, as customers watched the proceedings with concern) gave more opportunities to rivals, Inflection’s CEO didn’t disagree.
“Basically, I’m building a business,” Suleyman said. “And it’s extremely competitive.” This is the most competitive and creative time Silicon Valley has seen in years.