2021-2030 is Going To Be a Wild Ride

GPT-3 and Perceived Exponential Technological/Scientific Progress

Recently, I saw this tweet from Matt Shumer:

… and it really connects with what I’ve been feeling lately too.

GPT-3, some have argued, passes the Turing Test … which may not be the best test for AGI, but it is still an exciting milestone for the field of computing overall. A lot of people who use GPT-3, like me, certainly believe something definitively changed this year for the progress towards AGI at least in some kind meaningful of way - even if we don’t fully understand it. If the computing power trends continue and we create larger transformer models with more parameters, it might become really, really, scary … the things GPT-X in the future could be capable of.

Quantum computing advancements, AlphaFold, and GPT-3’s performance on zero/few shot NLP test benchmarks are all things I thought were years away, but here they are … today!

And so it leads us to the larger question - are we starting to really, really, feel the exponential nature of technological and scientific progress perceptually? Yes, it’s well-known that humans have a bad intuition overall psychologically for exponential curves. But what’s interesting is this year certainly felt more exponential to me. Far more exponential than previous years - but this is a very subjective thing to say.

If this rate of progress continues to surprise me year over year, what does this mean for this decade to come? Should we get comfortable expecting these kinds of breakthroughs on a regular, perhaps, monthly basis even? Well, the math says yes and I guess my heart raises that further and says, “hell yes”. So, I guess let’s sit back and enjoy this ride. Let me know if you felt this excitement about exponential progress in 2020 too.