I was absolutely willing to share my experience with GPT-3 and what I know. I’m glad he asked because these are some pretty important questions that I think lots of people have about GPT-3.
Today, I’m sharing my discussion with Harish from this interview.
Let’s start with some background. Who are you and what is your backstory?
My background is in web development. I spent years building an independent web search engine which I successfully brought to market. I had a shoestring budget, but still managed to crawl the web (over 250 million websites), data mine, index the content, and build the frontend/backend interface for searching the information. It was like a mini-google, without 1,500+ engineers behind it. To my knowledge, I'm the first person to ever accomplish such a feat. I was proud of that experience and it continues to inform my work with GPT-3 everyday.
Besides coding/tech, I'm just a creative person overall. I love making videos, writing, I used to make music, and recently I've been getting into "speed painting" too on my iPad. I run a YouTube Channel, a newsletter, and I love both of these things because they provide an outlet for my interdisciplinary nature with my passion for GPT-3 and AI technologies. It's a joy of mine to connect with others over GPT-3 especially and hear all the different ways they are using it.
Ideologically, I'm an intense idealist and futurist. I sincerely believe technology can solve every problem in the world, which is what energizes me constantly. I am passionate because I believe in the importance of building a Utopia. Everything is achievable through technology ... I love this industry.
What have you been creating with GPT-3?
Mainly YouTube videos. Some of my videos I'd say are more technical like when I discovered GPT-3 can not only write functional code, but obfuscate it too. I have other technical GPT-3 videos as well like on tokenization or N-gram generation which were interesting discoveries.
Besides that, I take pride in my videos which explore the human effect of GPT-3 as well. For example, I made a whole video on how GPT-3 gave me a literal identity crisis after generating tweets which sounded just like me. This video is a part of a larger series I'm working on called GPT-3 and Identity. There's already a follow up video on how GPT-3 later gave me a hero-identity crisis as well through a real person named Naval Ravikant.
What interested you in creating with GPT-3?
I've always had an obsession with AI even as a kid. It was a topic I would tell all my friends at school about and was disappointed to hear they knew nothing about it and weren't interested. Later, I remember being in university and having to explain to people what AI even stands for and it was really disappointing to me personally. How do you know who Taylor Swift is, but not what AI is? Anyways, I have been following the state-of-the-art of AI for years now, but GPT-3 is so compelling this is the first time I really wanted to jump in and participate.
Have you had other exposure to AI tech in the past?
But everybody has had exposure to some extent, my earliest exposure was the movie A.I. Artificial Intelligence by Steven Spielberg. Likewise, if you've ever seen any of the Avengers/Marvel movies with Iron Man, he has Jarvis (who later becomes Vision) which is an example of an idealized artificially intelligent system.
The cool thing about GPT-3 is you don't need exposure to AI in order to use it. You don't even need to know how to write computer code. It sounds crazy to most people, but the most shocking thing about it is that you can just communicate with it in English (or most other languages) and a lot of the time it just, "gets" what you mean. It's really that simple and usable for such a sophisticated and powerful system. It's a real breakthrough that you can just use natural language, talk to a computer, and have it do stuff for you. This has been the dream since the beginning of the computer industry and I'd say this breakthrough is here, today.
It's worth noting, I believe in building a utopia, a world of abundance and no resource dependencies. Why do we fight each other as nations over trivialities that happened decades ago? Why do we let people starve? Why do we have mental health issues? All the world's problems - mental health, world hunger, disease, equality ... these could all potentially be solved through AI. I strongly believe everyone should have an interest in AI and take it more seriously as a viable solution to the world's problems.
What are your thoughts on the risks of building with a closed Platform like OpenAI? What can product builders do to mitigate those?
I would push back on "closed platform" a bit here. Apple is, "closed", right as a company ... do we agree on that? But OpenAI often open-sources key software system assets (like CLIP), shares their research findings (in a pretty detailed way) as they happen, and has made their core technology accessible through an API (despite the safety/legal risks as a company of doing such a thing).
At the same time, we don't know if Google or Apple or any of the other big tech companies have their own top-secret large scale transformer models or what they're doing with it. They have certainly not made it accessible via some kind of API. Would you describe them as, "closed" too, in the same bucket as OpenAI? They have far more money and talent and could absolutely spin up GPT-7 right now. That sounds a lot more closed to me.
Also, yes, GPT-3 access is still an issue, it is not widely available yet and it is still in private beta because of the safety risks of opening it to the public. But, I recently spoke to OpenAI and brought up access and transparency concerns on behalf of the community and they were quite receptive to my feedback.
If you want OpenAI access and are having no luck with the waitlist, I encourage you to watch my video on how to get GPT-3 access. To my knowledge, it is the best (unofficial) resource out there on how to get OpenAI's GPT-3 access and I'd estimate that it has helped at least hundreds of people get access.
To answer the larger question - there is always a risk to building your core business around someone else's API. It's kind of like building on rented land. However, OpenAI is the only game in town. I've heard GPT-3 cost millions of dollars to train, there is no one else who is offering access to such a powerful model through an API. Unless you have millions of dollars to spare, you're going to have to work with them and honestly, they've been pretty friendly, open, and supportive at least to me so far. So, I'm happy with the partnership and I think you will be too. Besides, I am interested in their feedback around safety anyways as they are the ones who built the model and know it the best.
Still, if you're not comfortable with this partnership/business arrangement, you can always fine-tune the previous GPT-2 model OpenAI has previously open sourced entirely and it might help you. You can retain full control that way and would not have to partner with OpenAI, but please know that GPT-3 is many, many, times better than GPT-2. I'd even say GPT-3 is light years better. So, in a way, I'm not sure if fine-tuning GPT-2 would be the best use of your time.
What is your advice for founders looking to build products with OpenAI API?
I made a whole article on how to build a startup monopoly with GPT-3. I encourage every founder to read it. GPT-3 based businesses do not have the same defensible core competencies compared to most businesses. This is a fancy way of saying that it's pretty easy to copy and compete with you if your business is powered by GPT-3. I encourage all founders to think through their "business moat" and how they are building a unique business which is difficult to copy and compete with.
Also, I'm working on a YouTube series on Altruism, so, while I'm answering this question, I encourage all founders to think of altruistic use cases as well for GPT-3 or how they can incorporate altruism into their GPT-3 based business.
What learning resources would you recommend to the founders looking to build AI-enabled products?
I recommend getting GPT-3 access. As soon as you have access and experiment with it, you'll have a better idea of what it's capable of doing. It's a magical experience too and will change how you look at the world and how things will evolve this decade.
If you have access, spend your time experimenting with GPT-3, asking it things, and incorporating it into your daily life. Be clever and use it to help you solve problems in your daily life. There is no learning resource that can prepare you for it. Pure experimentation and playing around with it is the way to go.
What is your take on the current AI capabilities and also where are we headed in the next few years?
I encourage everyone to read OpenAI's scaling law papers. In my opinion, they are sort of the "holy grail" of the current state of the art in AI.
Personally, I'm excited about "multimodal" models which are coming next. Unlike GPT-3 which only does text, these models are trained and can generate images AND text. Or text and audio. Or even text, audio, and video. I encourage you to check out the blog post on DALL-E, which is OpenAI's upcoming - sort of - sister model to GPT-3.
My next YouTube series will be called, "GPT-X, DALL-E, and our Multimodal Future". It explores the effects of multimodal models, particularly on human creativity and creative works.
Where can we go online to learn more about you or what you are creating?
Sure - if multimodal sounds interesting I encourage you to join a free subreddit resource community I created called r/multimodal.
I encourage you, again, to watch my YouTube video on how to get GPT-3 access.