GPT-3/Multimodal/OpenAI Predictions for 2022
OK, here we go.
Honestly, since August of this year, I have been speculating in the back of my mind about what is next for the GPT-3/OpenAI/Multimodal space in 2022. Some things are pretty obvious just based on the existing trajectory of today’s products, other things may be rumoured that I can see actually happening, other things we can draw reference from the OpenAI scaling law papers (and follow up research papers by others), and finally other things just have to do with me speculating heavily based on different mental signals I have accumulated over the years about what is going to happen next.
I want to be clear - a lot of the time my predictions are super wrong or very early/optimistic, but other times I can be really right. Regardless, you should almost treat everything in this article basically as fiction and of course, I always encourage you to please share your ideas and thoughts in the comments … especially if you feel there are gaps in my thinking, technicalities I am missing, and things I could learn. I certainly don’t know everything and my experiences as one person are indeed limited.
I want to first address one question - why bother speculating about 2022 anyways? Does it really matter? None of us are looking at OpenAI or Google’s balance sheet, private product roadmap, or even their cloud console hourly billing fees. At the same time, there is this idea of whatever is going to happen, will happen … regardless of what an armchair speculator like me has to say. So, is this just an exercise I’m doing for internet points? Is this “what to expect from 2022” format just something I’m stealing from other industries because they have realized people love reading this kind of content marketing junk on their phones during the holidays instead of spending, “quality time” with their family?
I want to assure you this is not the case. I like talking about the future because I deeply feel the future is important. I also do try my best to talk about things in concrete ways. Also, when it comes to things which are further off, I still like even putting the ideas out there … out into the ether … because I feel it does give them some chance of actually becoming a reality. I’m not sure sometimes where the lines are drawn between science fiction and actual technical product roadmaps, between reading something I wrote and actually implementing it in real life at the company you work for. However, the point is, I’ve always believed that the thought of something being possible and putting it out there, is a necessary step to it actually becoming a reality. At the same time, I also make predictions which may raise the alarm about upcoming challenges - this is to get the conversation going and put pressing ethical challenges onto peoples’ radar.
Maybe there is also some “fantasy football” element too which I can’t explain. I’m talking about calling out predictions in advance and seeing if they actually happen. Also, it is fascinating to swap prediction notes with other people. Probably not the best motivation, but I’ll acknowledge this.
But really, I think this is an important space with very high stakes that we should be very thoughtful about and also these kinds of prediction exercises do help align people as an industry about what’s important and what the priorities even are. It helps to chart out some game plan for your company or even your career by at least having some kind of idea about what may happen over the next 12 months or so. Personally, putting stuff out there also helps to inform my own worldview about the future as well, based on feedback from others, which can be deeply flawed in so many different ways.
Anyways, I guess all I’m saying is that even though I think openly discussing the future is an important exercise, please take the predictions below with a grain of salt. None of this is based on inside information that I exclusively have. Anything I’m guessing which could happen could also be derailed - literally - in the first week. The game is constantly evolving and any party of any size could make meaningful contributions at any time … which would change pretty much everything for everyone involved. That’s part of the fun of the tech industry and the era we live in. Also, if I don’t get another chance to say so, I wanted to wish you an awesome 2022 and thank you for sticking me with this year.
Without further ado … the predictions …
My 2022 Predictions (in order of somewhat feasible to unlikely)
OpenAI’s 2022 Strategy - Increased Developer Satisfaction, Offer Outsized API Benefits
OpenAI in 2022 The “No Brainer” Language Model Company of Choice
Due to increased competition as well as OpenAI’s long term strategic goals, my guess is that they will focus on making the existing OpenAI Beta API Program even more of an obvious API partner choice. I think they will double down and offer outsized kinds of benefits that others simply do not or cannot even offer. If I was them, I would want to make the OpenAI API something of a no-brainer for anyone choosing a language model API in 2022.
I tweeted recently what parts of this could look like (tweet thread, click for more):
Reduced API Pricing
I think the big piece they’ll do next year is reduce the API pricing by something like 10X the current rates. I’m not going to comment on the safety risks or infrastructure costs of doing such a thing, but I do think reducing the pricing every once in a while is a very pragmatic thing for them to do. I wouldn’t be surprised if their financial models show them that they make more money long term this way and also increase people’s dependency on the API. Nonetheless, I could be super wrong on this pricing thing, so, please don’t quote me on this (unless it actually happens).
Streamlining Fine Tuning
At this point, I’d say fine tuning is strategically critical for OpenAI, so we’ll see a lot of improvements, updates, and usability enhancements in this area likely in 2022.
OpenAI App Store
I think it would be cool if OpenAI launched some kind of app store, or really app showcase, which includes different GPT-3 products and services which people can check out. They may already be working on something like this.
Business as usual
Don’t get me wrong, I think they will still continue adding value to their product lines in the existing ways they have already been doing so this year. This includes:
improving technical documentation
scaling infrastructure, global availability
streamlining the app commercialization process
working on setting industry leading technical benchmarks for specs like inference speed and latency
adding new API endpoints such as the embeddings API we saw a few weeks ago
engaging more with the community
UI improvements to the playground and other areas of the site
Increased sales and marketing activities
Updates on GPT-3 engines which summarize books as well “Math GPT-3”
More OpenAI GPT-3 Competitors
This year, we saw new competition enter the scene. Everyone from EleutherAI, Microsoft Azure, AI21, Cohere, and even Cedille. I see no reason why this trend would not continue well into 2022, giving developers more options and value offerings. It’s possible we may see the existing big players like Amazon step in as well. Google/DeepMind has made announcements in this area but, perhaps, in 2022, they may also open up a language model of their own to the broader developer community like OpenAI currently offers.
2022: A Critical Year for Online Communities and Discourse
I’ve already published a whole piece on why I think large scale language models (not just GPT-3) will start to stress test and potentially put a strain on the internet’s existing community and social media infrastructure next year. It’s an important problem with serious implications.
My Biggest 2022 Prediction: GPT-3 will take over schools and college campuses
I just published a piece this week on why I think GPT-3 may sweep college campuses in 2022. It’s a really exciting possibility with the potential to dramatically reshape the educational system in foundational ways.
More Verticalized GPT-3 Engines (GPT-3 Legal, Customer Support, Medical, Email, etc.)
I already mentioned Math GPT-3, GPT-3 for summarizing books, and of course OpenAI codex. I think we’ll see some official, specialized GPT-3 models for even more new verticals such as:
GPT-3 for Legal Text Analysis - this one brings with it many ethical/safety challenges, but there are definitely pockets in the legal world in which this could be commercialized and implemented
GPT-3 for Customer Support/Chat Bots - I would imagine many fine tuning jobs are ran just to create safer and professional customer service bot agents … I could see this being an area of interest for OpenAI. The idea would be that you have some base model specializing in customer support and from there you could fine tune with an even smaller subset of your own data for your business, saving you time.
GPT-3 Medical + Email - The email and medical domains could also be really valuable for OpenAI to serve through specialized GPT-3 models
GPT-3 for kids (?) - This one is super risky, but I wouldn’t be surprised if OpenAI releases something like this maybe in a few years. The idea is to expose kids to GPT-3 sooner in a highly controlled and safe way. This would open up a whole new industrial safety standard and a whole new class of GPT-3 powered educational apps. A model which could actually do this successfully could serve as a really important safety milestone for the entire industry.
Addressing the 2021 GPT-4 Rumour Mill
There was a leaked AMA thread with Sam Altman (CEO at OpenAI), which has since been discredited and taken offline. I’m not going to be linking to it here, but I’ll summarize some of the main points which were allegedly said, which I read from it.
GPT-4 is coming
Apparently, GPT-4 is coming. It will have the same number of parameters as GPT-3 but way better performance since it’s based on better training data. I have already released a post suggesting GPT-3 could use a training data refresh anyways (it’s about time, don’t you think??).
Codex 2.0
Apparently, the next version will be far more impressive (and scary) as a code assistant
DALL-E Updates
No timeline on this, but we may see some updates on DALL-E/multimodal models. I think the thread said that OpenAI will eventually make all of their models multimodal, please don’t quote me on this. This is serious hearsay.
Anyways, maybe the stuff we saw last week for GLIDE is exactly this type of DALL-E update we’re looking for.
GPT-4 = New Terms of Use Policies? New Use Cases?
I’m going to throw this out there, but I’m not sure if another major version of GPT-3 will lead to a different set of terms of use policies and documentation. Are we scaling up safety risks as these models become more powerful? I would assume so. When I really think about this, the funny thing is, I can see a reality where OpenAI’s technical documentation is actually segmented by GPT version (a lot like other kinds of technical docs).
It’s worth noting, I’m especially interested in what new types of GPT-4 use cases will be possible. I’m sure the existing use cases will perform a lot better, but again, what new capabilities will GPT-4 have that its predecessor did not? This is a really important question and may be a lot harder to answer besides just trying different things in the playground.
Even more Copy.ai Competitors
This is something I don’t quite want to see happen, as I am friendly with the copy.ai team, but I think it is possible a big company like Adobe, Salesforce, Canva, or Hubspot begins to offer copy.ai’s functionality as a value add to their existing service. Perhaps, they could even offer it entirely free as a part of a broader content marketing/lead generation strategy. I’m sure copy.ai already has a lot of smaller competitors, but the bigger ones are a potential threat too. It’s important to keep track of what may happen to them, as these implications could also affect other GPT-3 based businesses in the future.
Upcoming Industrial Criticism - Environmental Impact of Language/Multimodal Models
I have already detailed in a piece which I wrote last week that I will be Offsetting My Own GPT-3 Carbon Emissions in 2022. Although I think most data centers already run on renewable energy, my fear is actually that environmental concerns will be politically weaponized as a tool to discredit large scale transformer models like GPT-3. So, I’m saying it may make some pragmatic sense to instill more environmental consciousness into the community in 2022 and get ahead on this one.
Geopolitical/Economic/Global Pandemic Concerns
From a completely macro perspective, I think it’s worth mentioning that, I do not feel optimistic about the geopolitical situation we are looking at. It’s looking like the US may possibly be facing a dual front war over over both Taiwan and the Ukraine at the same time. Besides this, Omicron could also set back more supply chain activities and negatively affect the global economy/financial markets.
What does all of this mean in our multimodal/language model world? In uncertain geopolitical times, there’s likely a greater risk of language models used as a part of mass propaganda and of course disinformation campaigns. I’m not too worried about this, to be honest, but it is a possibility.
Sorry to end on a such a gloomy note. I honestly hope I’m wrong on that last one. Thank you again for reading my predictions, happy holidays, and I’ll see all of you in 2022!