"Creativity is just connecting things."
- Steve Jobs
People are currently going crazy over ChatGPT gaining the ability to talk, see, and hear. This ability is obtained from a combination of multimodal models and modules that OpenAI had previously built. While this is a great feat, it shouldn’t be unexpected. OpenAI has had the components to do this out in the open for over a year. It just took some time to glue it all together. When you have lots of components pre-built, your speed of creativity can accelerate.
Let’s take a look at what OpenAI needed figure out how to glue everything together to give these new senses/abilities to ChatGPT. CLIP (seeing) was the first multimodal model released which can connect images and text. CLIP was released in early 2021. Whisper (hearing) is one of the best speech-to-text models available and was publicly released in September 2022. If you can do speech-to-text, it isn't a far stretch to run the architecture in “reverse” to do text-to-speech (talking). The GPT4 large language model provides a great backend for understanding the language/text in both CLIP and Whisper. It acts as glue for the new ChatGPT capabilities.
Having these components built ahead of time enabled OpenAI to quickly create even more advanced abilities. But why could they be more creative, producing things that weren't there before? Creativity is as much about combining the things around you as it is seeing things that others haven’t. As Steve Jobs said:
Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn’t really do it, they just saw something. It seemed obvious to them after a while. That’s because they were able to connect experiences they’ve had and synthesize new things. And the reason they were able to do that was that they’ve had more experiences or they have thought more about their experiences than other people. Unfortunately, that’s too rare a commodity. A lot of people in our industry haven’t had very diverse experiences. So they don’t have enough dots to connect, and they end up with very linear solutions without a broad perspective on the problem. The broader one’s understanding of the human experience, the better design we will have.
Think of components as experiences. The more components you have in your collection, the easier it is to create new things and the more innovative you can be. It’s worth spending time to build components or modules that can be reused, so that creation can happen faster. Thinking and planning for future combinations can be a powerful thing.
Combining
The act of combining multiple elements is ruled by an area of math called combinatorics. To compute the number of possible combinations you need to know the number of elements, n, and the number of distinct elements you would like to combine, k. This can be written as
The thing about combinations is that each additional element adds an exponential number of possibilities. If you choose more than two elements to combine, the number of possible combinations increases at an even faster rate, up to a point. This behavior can be observed in Figure 1 below. Note that increasing the number of elements selected increases the number of combinations, but only up to about half the total number in the set and then starts to fall back. The maximum number of combinations for n elements reaches its maximum at ~k=n/2, and there is only one combination for k=n.
Figure 1. Understanding how the number of combinations changes
Looking at the plot in Figure 1, think from the context of building a new product. What is interesting? You don’t need a lot of elements/components to have a large number of possible combinations. Also, if you can think of ways to combine more than two components, the number of possibilities gets even larger. Now not all of these combinations can be meaningful or solve your problem; you will obviously need to filter out most of them. But if you are diligent in your creation to avoid overlap while steering towards a common endpoint, the serendipity can surprise you.
Building
If you are building a product or team, particularly in a fast moving field like AI, you want to accomplish objectives quickly while also creating optionality in order to take advantage of unknown situations in the future. Thinking in terms of combinations can help here as it maximizes your creative surface. Successful managers should be deploying a strategy to take advantage of compiling multiple components together and having their teams operate in a similar manner. This enables teams to snowball their way to success as seemingly random events shape the future. Let's look at some strategies with this mindset for building products and building teams.
Building Products
Any manager worth their salt breaks down a larger goal into smaller, more concrete objectives. The two key corresponding questions are "how should these objectives be split up?" and "in what order should we tackle the objectives?". In our fast-pace industry, it is very beneficial to order things such that you can accomplish things quickly and show proof of progress. This reduces to a snowballing strategy as follows:
Take your larger goal and break it down into smaller, achievable objectives
Each objective should be structured such that the end result is a component that can be used somewhere else in the future. It should not be just a singular accomplishment.
The mindset for component building that comes out of each objective is that if you had to walk away from the overall project at any point, you would still have components that you could use on other projects. Additionally, this helps to minimize the risk of loss.
Ideally, the objectives are ordered such that each one takes a similar amount of (short) time to accomplish. The caveat is that the reduced time for later objectives is only possible by relying on the use of components that were previously built.
Lining things up in this manner creates an accelerating feeling of momentum both to the internal team and to outsiders as you are able to accomplish larger and larger feats in a similar amount of time.
After you reach your objective, pause and look at all the components you've built to see what can be combined in previously unthought ways to be more innovative or accomplish even larger goals.
The strategy above should give you a way to be creative in chaotic environments. If you look at OpenAI, they built their GPT system and that was a stand alone system for text generation. Then they built CLIP which allowed them to do text-to-image creation. Then they built Whisper to do speech-to-text. Then they built ChatGPT to be a better mode of interacting with GPT systems. At each point, these prior systems were their own product. Then OpenAI stood back and combined all of these components into a single system. OpenAI was accomplishing an objective with each component while simultaneously building towards a bigger goal. Then they creatively combined all of the prior components into an even better product.
Building Teams
From a team development perspective, a similar approach can be used. What I advise my team members to do is to collect useful code and procedures they develop so they can build up a toolbox they can pull from throughout their careers. I also recommend that after they complete a project, they take a step back and look at what modules they wish had existed before they started the project, and then I advise them to go build those modules. This allows them to solve future problems more quickly and creatively by leveraging what they've built in the past. It has the added benefit of being able to tackle harder problems more readily. Taking on this habit creates a mindset of growth and trains individuals to be on the lookout for ways to make their toolbox better. It also helps individuals be more creative by seeing combinations all around them.
It's hard to predict what will happen in the future. That's why it is useful to build things that will be useful and valuable regardless of how the future shakes out. The more components or things you have, the more you can combine them in interesting ways. As Louis Pasteur said "chance favors the prepared mind."