Coping with the Pace of AI
Remaining Stoic in the Changing Times of AI
Faster, Faster, until the thrill of speed overcomes the fear of death.
-Hunter S. Thompson
Ever been in a hurricane?
How about a stampede?
That's how a lot of people watching the progress in AI feel right now. Over the past few weeks I've spoken to a lot of people outside and inside the AI industry to get an understanding of how people are feeling about the pace of AI. Some are freaking out. Some are excited. Most are overwhelmed. A select few don't think things are progressing fast enough. Even those who have been working on large language models (LLMs) for the past 3-4 years are still overwhelmed with all the announcements. To get a better understanding of what's going on let's explore a few topics: filtering information, controlling reaction, and what you can do.
Filtering information
Let's be realistic - the genie is out of the bottle and the world is indeed changing due to AI. The way people work is about to dramatically change and that shouldn’t be taken lightly. We are just at the start of a Cambrian explosion of AI, where the amount of technology and products will be released at a mind-boggling pace. In fact, Bill Gates recently stated that the Age of AI has begun. He writes:
I think back to the early days of the personal computing revolution, when the software industry was so small that most of us could fit onstage at a conference. Today it is a global industry. Since a huge portion of it is now turning its attention to AI, the innovations are going to come much faster than what we experienced after the microprocessor breakthrough. Soon the pre-AI period will seem as distant as the days when using a computer meant typing at a C:> prompt rather than tapping on a screen.
Knowing that this deluge of AI will become the norm, we should, for our own sanity, separate out what we are hearing into three buckets of increasing frequency.
Technology improvements
Applications of a technology
Observations about what can be done with a given technology.
Technology improvements are core advancements in what AI can do. By definition technological improvements happen at a slower pace than the other two categories of information. The creation of transformers and reinforcement learning from human feedback (RLHF) are recent core concepts that would fall into this category. The release of new foundation models like GPT-4 and Midjourney v5 would also fall into this category. The key understanding here is that announcements in this category will be of lower frequency but higher impact. Whenever there is a new announcement in this category, expect a jump in announcements in the next two categories.
Applications of the latest technology (including product releases) use underlying technology improvements to create something new. A lot of the conversation on AI progress you are seeing right now is about the breadth of application that people are finding and less about the technical progression. Yes the technical progress enables these applications but bringing new tools to life tends to resonate with people more. This category ranges from medium to high frequency with similarly medium to high impact. ChatGPT plugins would fall into this category as they are an application of two technologies - LLMs and APIs, combining the two in a somewhat novel way. Curiously, the essence of ChatGPT plugins were created at a hackathon about two months before the plugins were announced by a project called “GPT is all you need for backend”. However, since DY has a much smaller reach than OpenAI and the project was not a fully-fledged product, the release was less well known.
Observations are a continuous stream of conversation about both technology improvements and application. These observations are either thoughts on the implications of a new technology or showcasing what a technology can do. When people show what they’ve done with ChatGPT, such as Bard vs GPT-4 in a rap battle, using GPT-4 to build a business, or how to take over Twitter, I call these observations. These individuals are using the technology where the usage is the end point and the results are provided for others to observe. Observations, while provocative or profound, tend to occur at a very high frequency but with low impact to actual progress.
Figure 1. How frequency of messaging compares to time to create an entity within the three filtering categories.
Summarizing how the three categories interrelate, we can plot them in Figure 1 on the time it took to create the entity vs the frequency of messaging you see on the topic. Typically things that take relatively longer to create tend to be more impactful. Viewing this plot should help you identify where to pay attention. If you understand what is important, it should be harder to become overwhelmed.
Controlling Reaction
Looking at past two weeks, it makes sense why many people have been so overwhelmed with the pace of AI. Figure 2 shows the timeline for only a slice of these announcements. Looking at these, they tended to be higher impact and announced at roughly the same time, with little to no time to digest and truly dig into any single technology or application.
What you don’t see is how much time went into making any of this technology or applications. You were suddenly hit with a continual firehose of high impact announcements. While it may seem that a lot of these technologies and applications came out of nowhere, they have actually been in development for quite some time. So, while a lot is being released, it doesn’t mean groundbreaking things take only two weeks to develop.
Figure 2. The recent release of a few AI technologies and applications over the past week that came at a furious pace.
This overwhelming feeling has caused many people to have a defeatist mentality. This typically sounds like “When AI is strong enough there will be nothing for humans to do” or “X industry is basically gone now”. I think this extreme, black-and-white type thinking tends to be inaccurate over the long term because it misses feedback loops and system dynamics. If you want to have good foresight, you need to have an understanding of how different players in a system will respond. People and companies don’t take things lying down. People will always find a way to use the tools available to them and adapt to new environmental conditions. If you are seeing great productivity gains in your work right now thanks to AI tools, enjoy the golden year you have using them. It’s only a matter of time before your required baseline output shifts to account for the usage of AI tools.
While AI is becoming highly impactful and changing how people operate, at some point it will become the norm of how society operates. We can work to embrace these technologies and utilize them to our advantage. AI may not be an expert in all things, but it might be better than you, and that might be good enough for what you need.
What to do:
Reduce information to first principles and system design. AI progress may be coming at a furious pace but having a solid grasp of the fundamentals allows you to put new announcements into the appropriate bucket. Do you understand transformers, embeddings, and RLHF? Then it becomes straight forward to understand the differences between GPT-4, LLaMA, and Alpaca. Understanding system design allows you to interpret how new functionality fits into current paradigms and gives you some foresight into what is coming down the pipe. ChatGPT plugins? They naturally fit into LLMs having actuators to acquire more information and interact with the world. They also fit into the mental model of GPT-like models being a new layer of computing.
Understand the rate of obsolescence or decay in the environment. At a rate of 3 announcements per week, if it takes you 1-2 two weeks to understand a new technology but that tech is surpassed in another 2-4 weeks, you won’t be able to keep up. By the time you fully understand and then build an application, something better has come along. Knowing this fact, don't let yourself be whipped around like a kite in a hurricane and adjust the time you spend fully digging into a new announcement until you actually need it.
Operate on a different timeline and focus on the long term. If you are being thrown around in a sea of fast waves, shifting the timescale you are focusing on can diminish the impact of hype and help you figure out what actually matters. Typically, viewing events from a longer time scale makes huge waves seem like ripples and makes it easier to comprehend all the changes happening in the environment.
Focus on solving your actual problem, not the hype of what's going on. Yes, a lot of the hype is real but that's because AI solves the problem of intelligence which allows you to solve any other problem. Now, tune out the hype. Instead, focus on if the current state of AI is able to solve your problem. Is the tech today able to solve your problem? Great, use it. Does it not? Figure out a way to make it better, improve the architecture, or apply it in a way that gets you most of the way there.
Don't be a passenger in chaos. Be proactive instead of reactive to all the AI news coming at you. Use these tools to develop a fundamental understanding of the new AI advancements and products work. Realize that what worries you is likely viewed from the lens of how you would use the system. Someone wielding the same tools won't see the same things or have the same ideas.
As people begin to use AI to create better AI tools, the pace of progress within AI is only going to increase. Having a stable head and using the tools above to quickly filter, sort, and compartmentalize announcements will hopefully help you cope with the pace of AI.