The Preoccupation with Optimization
Dealing with an increasingly changing world
For years the world has been moving in the same stylistic direction. And it’s time we reintroduced some originality.
- Alex Murrell
One of my favorite algorithms is the multi-armed bandit. It effectively solves the problem of: if you were to go to a casino with many slot machines each with a different payoff, how do you trade off exploration vs exploitation in order to maximize your total winnings? This balance between exploring the different possible solutions and optimizing for the best one is what makes this algorithm great. You would be surprised at how many different problems can be modeled in this fashion. In fact, there are so many ways to apply this algorithm that it was mentioned in the book Algorithms to Live By. If applied properly, the algorithm can even adapt to shifting distributions (say the slot machines being switched out). However, unlike the multi-armed bandit, one problem I’m seeing in the world is that as a whole we’ve been stuck in a routine of optimization. We’ve optimized so much that we lack originality and take on a higher risk of system wide failures. AI has likely contributed to this problem and if you are not designing it to adapt to changing conditions, you are likely doing more harm than good. I believe we need to get back to more exploration to make our society robust.
Over Optimization
Businesses crave reducing uncertainty. They yearn for a world where everything is easily projected with no volatility. Unfortunately, that's not the world we live in. We live in a messy, messy world where even if you are generally good at predicting, you can still make poor decisions. That's because behaviors and the state of the world change. That's where over optimization is dangerous. You can look at the supply chain difficulties that occurred during COVID to understand how over optimization can be painful. There's a similar concept in machine learning called overfitting. Overfitting is a bad modeling practice wherein the model has been anchored too hard to the existing data that it does not generalize at all to new situations or data. What's the simplest way to internalize overfitting? Look at the picture below of a bed. A bed shaped as a person curled up is pointless if you wanted to sleep in any other position.
Figure 1. The simplest explanation of overfitting I've seen.
Where I’m seeing a lot of issues is in people thinking that yesterday will be just like today. They build solutions that optimize for yesterday as much as possible because they believe it will be the most efficient and therefore maximize profit. However, as a you optimize a process more, there is less room for error and any misstep has outsized impact that can reduce months or years that were saved from those same optimizations. It’s unfortunate that people chase more and more optimization when we know the world changes. In machine learning there’s a concept called model drift, where a model’s performance worsens over time and the model needs to be retrained. This happens because users are interacting with the model outputs and their behavior changes. The data of the users becomes different from what the model was trained on, and the difference causes inaccuracies. As practitioners, we know this happens and we put processes in place to account for, monitor, and control for this phenomenon.
When applying AI, you might get asked to “find the optimal solution”. Your first question should be “optimal to what?”. Optimal to the current data? To the current state of the world? To where the world is headed? To maximize profit? To minimize loss? To be robust to changing conditions? What feels like a straightforward request has a lot of nuances that greatly affect the outcome. Nuances that make the difference between a stellar solution and a great deal of pain.
Over-optimization leads to very fragile systems. Systems where small errors and mistakes, which are bound to happen, can greatly impact the system more much than expected. Nassim Taleb writes in his book Antifragile about the differences between fragility and anti-fragility.
Simply, antifragility is defined as a convex response to a stressor or source of harm (for some range of variation), leading to a positive sensitivity to increase in volatility (or variability, stress, dispersion of outcomes, or uncertainty, what is grouped under the designation "disorder cluster"). Likewise, fragility is defined as a concave sensitivity to stressors, leading to a negative sensitivity to an increase in volatility. The relation between fragility, convexity, and sensitivity to disorder is mathematical, obtained by theorem, not derived from empirical data mining or some historical narrative. It is a priori.
Initially, there is no harm in finding something that is optimal. However, the idea that one can pursue a single optimum that will perform well is a fallacy. What does this pursuit lead to? It leads to people doing more and more of the same until what was thought to be great is average. Alex Murrell writes about the "Age of Average":
What he found was surprising:
“Until the year 2000, about 25% of top-grossing movies were prequels, sequels, spinoffs, remakes, reboots, or cinematic universe expansions. Since 2010, it’s been over 50% ever year. In recent years, it’s been close to 100%.” […]
“In 2021, only one of the ten top-grossing films (the Ryan Reynolds vehicle Free Guy) was an original. There were only two originals in 2020’s top 10, and none at all in 2019.”
Reusing the same message becomes tiring, ineffective, and potentially damaging. You play baby shark 1 time and your kids love it. Twenty more times and they have fun. Play it a thousand times and parents are on tilt. Play it 24/7 and you are in court for torture. Converging on the same message as everyone else will is non-optimal even if the historical data points to similar messaging as others. This is what message convergence looks like:
Figure 2. Brands converging to the same look and feel
Improving Exploration
World states change and distributions shift and what was optimal before no longer is. We are in one of those times right now. Many people are focusing on how to do what they are currently doing faster, rather than focusing on the new things they will have to do if they really want to be "optimal" near term. To adapt to changing conditions you need to explore. To get around blandness, average, and systematic risk, one needs to have an exploration budget. A budget for exploration is an amount you are willing to spend to find novel solutions with high payoffs or to be robust enough to bad outcomes. Yes, spending this exploration budget means you are not being as optimal as you could be but it is worth the cost. Einstein worked in a patent office so that he could spend more time thinking. Darwin apparently worked only four hours a day to give more time to thinking and chance discovery. Imagine if they had been focusing on optimizing their day as much as possible for productivity. I doubt they would have made their massive discoveries.
Most of what I hear about how people are thinking of using AI, is to do things faster. This is expected but at the same time it is less interesting because it will quickly become the norm. Doing something faster becomes the new baseline. Don't get me wrong, I work in AI and I think the whole field is fascinating. What I'm interested in is, what does this new technology allow us to do that wasn't possible before it existed. What upgrades and new capabilities are we getting as a society from having this new technology? Hyper-personalization, which I wrote about a few weeks ago, is one of those things. Before generative AI, it was largely unfathomable to expect someone to create a mass-produced experience that is unique to a single individual. This was not just a case of being able to do more faster. Generative AI took something that was uneconomical and made it very valuable. If we focus strictly on ways to optimize what we are currently doing, we won't be able to spend enough time exploring to find the things that weren't possible before given technology existed.
When people first started using the internet, they thought it would be a great way to store and retrieve information. Not that we would live most of our lives there, communicating, using social media, and being device obsessed. For those of us that lived through the explosion of the internet, it was a time of mass creativity and exploration as people were figuring out what could be done from a vast network of information. People were more focused on figuring out what was possible rather than figuring out what was optimal. I believe the era we are entering requires us to again adopt the mindset of exploration. So when we think about the pervasiveness of AI, what are some examples, concepts, and questions we should use to think about what the world might look like?
What are the relative differences between different levels of AI? How do they shape our society?
What used to be cost prohibitive but is now economical by using AI?
What happens when everyone has a personal agent or assistant that accomplishes tasks on their behalf?
How much value can accrue in the ability to upgrade one's agent?
Will people spend more or less time with technology if they can offload tasks to agents?
Red Queen Effect - if everyone adopts AI into their work, what becomes the new competitive advantage? Alternatively, how long will the current gains people are seeing last for?
How do our lives change when we can request a new movie completely unique to an individual in minutes?
What becomes the next time suck if people stop scrolling through social media? Does freeing people up allow them to engage more with each other or with nature?
Do people feel more or less free and in control of their own destiny as the mundane is offloaded to AI agents?
How much faster can we accelerate knowledge acquisition when AI can summarize and personalize information in the way you learn best? Does this mean children could learn calculus by the fourth grade? How does that shape the world? Do we approach Ender's Game?
How much can we improve our rate of scientific discovery when we can quickly eliminate dead ends with AI?
Thinking through how the world might change beyond the first order requires having an understanding of how different mental models interplay. I've found using multiple frameworks can also help understand where the future is going. One of those frameworks comes from Joi Ito who headed MIT's Media Lab and wrote a book called Whiplash: How to Survive Our Faster Future, wherein he laid out nine principles:
Emergence over Authority
Pull over Push
Compasses over Maps
Risk over Safety
Disobedience over Compliance
Practice over Theory
Diversity over Ability
Resilience over Strength
System over Objects
What all of these principles have in common are ways to successfully navigate a rapidly changing world. Joi attempts to provide frameworks and modes of operating that can adapt to volatile conditions. Conditions where if you focus on optimization, it will end poorly for you. This means dealing with the situation in front of you from the principles that govern the world instead of over optimizing on the current state of the world and hoping things will remain the same. Remember, there are a lot of stories about explorers and exploration, but I haven't read too many stories about those that were superior optimizers. Be an explorer.