An OpenAI Story: From Hegemon to Underdog
How one of the most successful AI companies lost an edge it may never recover
OpenAI was the very image of AI.
A company with a grandiose mission, a visionary CEO, a history of influential research, and a billion-dollar backup—it had the world at its feet.
Now, it is but a shadow of what it was two years ago.
Since then, innumerable startups have emerged. They’re young, full of energy, and well-adapted to openness and fast-paced research.
They’re also effectively threatening OpenAI’s reigning: the community is about to relegate the once hegemon of AI to a mere underdog.
Promises, ambition, and controversy
It wasn’t always this way. There was a time people liked OpenAI.
Musk and Altman founded it in 2015 as a non-profit research institute to ensure the safest development of artificial general intelligence (AGI).
Safety and alignment were their top priorities, existential risk (x-risk) their main worry—and DeepMind the only threat to their mission.
The AI community saw OpenAI as an honest endeavor—even if not everyone shared their perception of x-risks being the most important or urgent and didn’t trust OpenAI. Still, the company pledged to be focused on ensuring that AGI “benefits all of humanity.”
A crystal clear long-term purpose, a handful of top-notch engineers, and a strong sense of being the chosen one granted OpenAI a bright future.
But, after a few years of newsworthy successes—Gym, OpenAI Five, and the first GPT—the company came to a crossroads:
On the one hand, the non-profit, transparent nature of the company ensured it was untied from interested shareholders that could divert it from its humanitarian mission.
On the other hand, OpenAI’s executives realized they needed much more money than they could possibly gather as a non-profit if they wanted to build AGI—although, once again, not everyone agreed.
They eventually decided that the critical bit was that they were the ones to build AGI—even if that meant accepting money from powerful players unmoved by OpenAI’s long-term goals but very much decided to get some benefits out of the deal.
In 2019 OpenAI entered the for-profit space and partnered with Microsoft for $1 billion in exchange for an exclusive license. As reporter Karen Hao (previously at Tech Review, now WSJ) explains in a brilliant 2020 article on OpenAI, “competitive pressure eroded [its initial] idealism.”
But money wasn’t the only reason why OpenAI’s popularity rapidly descended.
Also in 2019, OpenAI hesitated to release GPT-2 as open-source claiming it was “too dangerous.” This annoyed many people: because the company had promised to be open and because it felt like such an obvious attempt at gaining visibility.
The company eventually opened the model but it was already marked by controversy.
The road to AGI is full of obstacles. OpenAI knows money, notoriety, and support are key factors in succeeding—but it should remember that ends don’t always justify the means.
What began as an honest and transparent endeavor soon transformed into a dubious strategy with rhetoric and hype as core tactics.
But OpenAI wasn’t done.
GPT-3: the power behind OpenAI’s monopoly
In mid-2020, with a few controversies in its back, OpenAI released its crown jewel, GPT-3: 100x larger than GPT-2 and capable of writing believable news articles, essays, songs, and poetry among other impressive feats.
Apart from being an unprecedented global phenomenon—magazines are still writing articles two years later—GPT-3 was a personal success for the company, proof that their beloved scaling laws worked: size is what matters. Bigger models—trained on more data with more computing power—are our best shot at AGI.
GPT-3 got OpenAI ahead of everyone else by a several-month advantage (Google was already playing around with these ideas but not with OpenAI’s conviction).
GPT-3’s unprecedented linguistic aptitude forced the AI community to pursue similar research directions with renewed intensity and interest. But OpenAI knew the training costs were high enough to prevent competition—only titans like Google, Meta, or Nvidia had resources to build their own GPT-3 which they wouldn’t commercialize because they didn’t have to.
OpenAI was alone at the top.
The company set up an API-paid model, successfully establishing a generative AI monopoly around GPT-3. They ensured those who wanted to benefit from GPT-3’s abilities would pay tribute.
But maybe they got too confident.
Open-source AI: An emerging trend and a growing threat
The community’s amazement with GPT-3 was only comparable to its annoyance at OpenAI’s radical shift from an open non-profit lab to a closed for-profit company in merely one year.
Building a business to earn money is licit, but it feels almost like treason if the original humanitarian mission suddenly turns into “I need money so I’ll lock my research to make a profit.”
EleutherAI was the first open-source initiative that stood up against OpenAI. In 2020, they set up the goal to build an open-source GPT-3. All the models they’ve released so far (GPT-J, GPT-NeoX, etc.) are available under free software licenses.
Although quality-wise they’re nowhere near GPT-3, EleutherAI had set a precedent: open-sourcing state-of-the-art (SOTA) generative AI research was possible.
The key insight was realizing that the tech itself wasn’t the bottleneck. Building GPT-3 is “simple and theoretically straight-forward,” says Connor Leahy, EleutherAI’s co-founder. “The only secret was that it was possible.”
The only obstacle? Money—in the form of engineering talent, high-quality data, and computing power.
Open-source initiatives with access to those would be unstoppable.
EleutherAI’s ultimate purpose came about in 2022, in convergence with Hugging Face’s and BigScience’s efforts. Together—supported by many more organizations advocating for open-source, open science, and ethical principles—built BLOOM: GPT-3’s open-source match.
As I wrote in June in an article entitled “BLOOM Is the Most Important AI Model of the Decade:”
“BLOOM (BigScience Language Open-science Open-access Multilingual) is unique not because it’s architecturally different than GPT-3 … but because it’s the starting point of a socio-political paradigm shift in AI that will define the coming years on the field — and will break the stranglehold big tech has on the research and development of large language models (LLMs).
…
[BLOOM] is the result of the BigScience Research Workshop that comprises the work of +1000 researchers from all around the world and counts on the collaboration and support of +250 institutions, including Hugging Face, IDRIS, GENCI, and the Montreal AI Ethics Institute, among others.
What they have in common is that they believe technology — and particularly AI — should be open, diverse, inclusive, responsible, and accessible for the benefit of humanity.”
EleutherAI and BigScience are what OpenAI wanted to be but failed to embody.
The text-to-image revolution: The final nail in the coffin
Neither the 2022 open-source crazy frenzy nor OpenAI’s descent stopped there.
An AI art scene began to emerge in late 2021 and early 2022 after a bunch of independent developers successfully tested OpenAI’s CLIP (a model to find the best image for a text string and vice versa) in combination with generative AI like GANs and diffusion models.
The bottom line: They realized it was possible to create amazing images, paintings, and illustrations using simple text commands.
Three reasons turbocharged the subsequent revolution of text-to-image models: The attractiveness of the visual component, the easiness with which anyone could leverage the models, and the relatively smaller size compared to their language counterparts.
AI art instantly became the most sought-after application in AI.
OpenAI once again played a key role: First, in 2021, with the publication of DALL·E and the open release of CLIP, and then, in April 2022, by showcasing the ability of DALL·E 2—at the time the unquestionable SOTA in text-to-image—released under a closed beta with an interminable waitlist (repeating the GPT-3 story).
But this time, the AI community knew what they wanted—and they already knew it was attainable. A contender that had been awaiting in the shadows since 2021 emerged just two days later: DALL·E mini.
(Funnily enough, Boris Dayma based DALL·E mini on DALL·E, not DALL·E 2, but it only reached notoriety after OpenAI released the second version a year later, overshadowing the achievement.)
Despite DALL·E mini’s lower quality, people soon forgot about the original. The reason? It was open-source. People flooded Twitter with the copycat’s creations and it went viral.
OpenAI surely felt the threat: it was a matter of time before a new company trained a competitive text-to-image model that disputed the company’s leadership in AI art as they were doing in language understanding.
Enter Stability.ai (for-profit, don’t get mistaken). In late August, the brand new company released the text-to-image open-source-licensed model Stable Diffusion—comparable to DALL·E 2 quality-wise but readily available to all through an easy-to-use website.
Stability.ai’s pressure prompted OpenAI to remove DALL·E’s waitlist (they did it two weeks ago). But now the question is, who’s going to pay OpenAI when we have free high-quality alternatives?
OpenAI knew its edge was temporary, but I don’t think they expected to lose it on both language and image generative spaces so early.
The current landscape threatens their core mission: if open source gets ahead, OpenAI may not be the first to build AGI—and thus no one will ensure it’s done safely and “for the benefit of all.”
At least that’s what they’re probably thinking right now.
But, how is open source eating AI?
Yesterday I came across a wonderful article by Swyx that perfectly complements this one. How is it possible that so many open-source alternatives, competitors—and by extension threats—to OpenAI emerged in such a short period?
Many factors are contributing to this inevitable—and irreversible—shift:
The release of open-source models by big companies (Google, Meta, and even OpenAI—although they never release their SOTA research).
The creation of open-source datasets like LAION-5B (LAION) or The Pile (EleutherAI).
The discovery of new scaling laws: DeepMind proved with Chinchilla (a 70B-parameters language model) that training data matters as much as size: the larger models—like GPT-3—are all undertrained, which means you can get better performance with smaller models.
The design of better hardware: Nvidia is constantly improving and optimizing its full stack and innovative startups appear every year with novel hardware, better suited (and sometimes cheaper) for AI workloads.
The cheapening of training: MosaicML has reduced the training cost of GPT-3-quality models to ~$500K. GPT-3 was in the $5-12M range.
The design of smaller models: Stability.ai is devoting efforts to reducing the size of models while keeping performance intact. Stable Diffusion can now run locally on an iPhone.
The discovery of better prompt techniques: Let’s think step by step, chain of thought prompting, ask me anything, and self-ask.
Many reasons are allowing much less wealthy companies to compete with OpenAI.
Three factors that put OpenAI’s business viability at risk
Here are three reasons why OpenAI should rethink its strategy going forward.
OpenAI is slow
OpenAI’s long-term goals—building an aligned and friendly AGI—require a constant safety assessment for every step they take. This contrasts and conflicts with open-source’s speedy nature:
If we assume OpenAI will be loyal to its original mission I can ensure you they won’t start releasing its models (as paid services) mindlessly. They believe control is significantly more important than openness because maximizing safety is the main criterion.
Stability.ai or Hugging Face think very differently: for them, safe openness > safe control. And I’d even dare to say that they believe unsafe openness > control. That’s the total opposite of OpenAI’s beliefs.
OpenAI has lost its competitive advantage
OpenAI’s business model was designed considering the company could hold a half-year competitive advantage over alternatives.
This was true when they released GPT-3 but they couldn’t replicate it with AI art models. Now, although inertia, publicity, and people’s reluctance to change are playing in the company’s favor, they’re probably earning much less money than, say, 6 months ago (both from GPT-3 and DALL·E).
They knew the models themselves were easily replicable. That’s why, before GPT-3, they tried to keep secret their knowledge that scaling size, data, and compute worked better than anyone else thought.
However, GPT-3’s results eventually spoke for themselves and everyone else began to copy their ideas.
Now, smaller companies like Cohere and AI21 labs offer services similar to GPT-3. Midjourney and Stability.ai offer services similar to DALL·E. Often—although not always—those services are cheaper.
Still, these companies are mainly VC-backed: can they build a profitable business before their accounts get to zero?
OpenAI depends on a weak business model
OpenAI is often compared to DeepMind, but they work very differently. DeepMind is a subsidiary of Alphabet (like Google) so they don’t mind spending a lot of money (much more than OpenAI) because they’re always backed by a super-profitable giant.
OpenAI is also compared to Google, Meta, and Nvidia. All those are players in the AI field, but OpenAI is much, much smaller than the others—in terms of employees and resources—and depends solely on the profits they get from AI-powered services.
Google and Meta sell ads and Nvidia sells hardware. They don’t care if open-source AI advances the field faster—actually, they may even benefit from it.
OpenAI’s rebirth?
OpenAI can still survive but its choices are limited.
Doing it DeepMind’s way
The easiest way for the company is to follow DeepMind’s steps and get acquired by a tech giant. Given that OpenAI has a partnership with Microsoft, that’s the logical choice.
They may retain some freedom, but their goals toward AGI may be subverted by Microsoft’s interest in earning profits from products and services.
Sadly for OpenAI—as far as I know—not a single big tech company believes in the importance of aligning with AGI as much as they do.
I ascribe a high probability of this happening in the long term.
If you can’t beat them, join them
If they want to keep pursuing their dreams, they can take another path: Join open-source. The release of Whisper is a step in this direction. They made the models, code, and weights completely available.
However, if the company does this with the next version of GPT and DALL·E, it’d endanger achieving its goal, as it’d eliminate all sources of revenue.
Given the history of the company of choosing to risk being influenced by shareholders before allowing others to reach AGI first, I don’t ascribe a higher probability to this option in any case.
Loyal to their path until the (new) edge
The last choice, and the one they’re most likely to take in the short term, is to continue the current path: open-source a few models—irrelevant commercially—and profit from the best they have.
Competitors would drive OpenAI to low margins, but if they manage to survive—and that’s easier when you have access to Microsoft’s money—they could simply wait until they take another leap over the rest of the industry.
That may happen with the release of GPT-4. If it turns out to be significantly and provably better than alternatives, OpenAI would, once again, effectively capture most of the market.
No one would use OPT, BLOOM, or even GPT-3 if GPT-4 turns out to be 10x better. That could give the company another several-month edge.
GPT-4 is around the corner, so we may discover how this story ends—or continues—very soon.
To be honest, your analyzes are MUCH better to read than others. Others' information is too dry. I'm so glad I found your newsletter.
Thank you very much!
I'm a relative newcomer to the AI scene, drawn into it mainly by my fascination with what it allows the average person (i.e. me) to do. Your in-depth takes are always a delight to read and give me a solid behind-the-scenes look at what's happening in the industry.
Just wanted to say thanks for yet another great piece!