Hype Deaths
Why causal analysis is hard; forgetting history
Hype is a term broadly used in common practice to describe over-exaggeration, distortion and unnecessary amplification of information associated with innovation. - Wikipedia
There’s a lot of hype surrounding AI right now, some warranted and some not, and it’s starting to feel a bit like a bubble. I recently spoke at HumanX about AI personalization. The general tone and mood of the conference was that while the adoption of AI is large, companies need to start showing a return on investment from AI technologies. To paraphrase two main ideas: AI is not a strategy, the strategy should be augmented by AI; and similar things have happened in the past. A potential signal of a hype bubble is when people only care about technology for technology’s sake instead of caring about how it can be used to solve certain problems. We can learn why hype happens by examining communication through different lenses. When people communicate there are differences between what they believe, what they want to happen, what they say, and how they frame things. The greater the difference between these lenses, the more likely hype is involved.
While it can be difficult to distinguish hype from reality without a lot of due diligence, the Gartner hype cycle is a great model about the hype state of various technologies. Tracking hype status with time, each hype cycle is broken down into five key phrases. First the technology starts with a technology trigger(1) and rises rapidly to the peak of inflated expectations(2). Then the downfall begins as the reality overtakes the hype of the technology, reaching a low point at the trough of disillusionment(3). From this point, there is a slight increase at the slope of enlightenment(4) and ultimately culminating in a stabilized state at the plateau of productivity(5). In layman's terms, technology is initially hyped to promote adoption and explore its uses, but when put into practice, technologies fail to meet expectations until they have been hardened and improved, at which point they provide value and ROI.
Figure 1. The stages of the Gartner Hype Cycle
Here's the 2024 hype cycle for emerging technologies:
Figure 2. The Gartner Hype Cycle for Emerging Technologies in 2024
You can go back to historical versions of this chart and watch how various technologies have progressed over time. You can also check how accurate it has been. What is clear from this chart is that the hype in every technology eventually dies. When the hype dies, the real work begins and a given technology rises to meet realistic expectations.
Outrageous Claims
You might have seen posts like this recently claiming the end of software engineering. They tend to use the chart below that shows the number of job postings for software engineering positions. They claim that AI has caused the demand for software engineers to drop by 70% in 18 months. These hype based click bait articles are pretty bad at analysis and flounder at causal analysis. Let's look at why.
Figure 3. The number of job postings for software engineers on Indeed in the US
First, what are you looking at? The FRED (Federal Reserve Economic Data) has a ton of data on various economic indicators. One of the indicators they started tracking several years ago was the number of job postings for many different types of careers. The chart above in Figure 3 shows an software development job postings index relative to when they first started tracking. You can think of this as a normalized count. As the index rises or falls it corresponds to a change relative to the baseline value when the index started.
Next, if you want to perform an analysis and put numbers into context, you need a second value. A number by itself is meaningless. A number in relation to another value tells you something. I could have $100,000,000 dollars but if the poverty line is $1,000,000,000, I'm still poor. If we want to understand the drop in the software engineering jobs index, we should compare it to the baseline of all jobs (Figure 4 below).
Figure 4. The number of job postings for software engineers compared to the total number of job postings on Indeed in the US
Well that's interesting. While the software engineering jobs have greater magnitude swings, the timing of drops is concurrent. Could there be anything besides AI that could have contributed to the rise and fall of that index? There are several macroeconomic factors that occurred before the decline in July of 2022, one being rising interest rates, shown on the the FRED graph in Figure 5. From this we can see that the decline in job postings started to happen just as interest rates started ramping up.
Figure 5. The number of job postings for software engineers compared to total job postings on Indeed in the US compared to federal effective interest rates
We can take this a step further and highlight a bunch of macroeconomic factors on the graph to demonstrate timing and effects. The point of this is to chart at when events occurred and observe what happened after. An event can't cause an action if that action happened before the event. To signify the differences, we'll put economic factors in red and AI type factors in green. AI has been around for a while (GPT type models have existed for over seven years), but for the purpose of the hype-death claims, we will define generative AI as ChatGPT onward. So let's mark the chart up in that fashion.
Figure 6. The number of job postings for software engineers on Indeed in the US annotated for key events
For reference:
WFH = work from home
SWE = software engineer
AutoGPT = first major AI agent library
Huh, that's interesting. ChatGPT came out when the number of software engineering jobs was already declining significantly. In fact, the index was through a majority of its decline before it was even shown that ChatGPT could generate quality code. AI also has no impact on the major change of slope. You know what does? When interest rates finally stop rising.
To understand the situation dynamics correctly, we should determine what caused the large increase in software development jobs before the decline. There are several things going on simultaneously. First, labor mobility (the ease at which people can change jobs) was very high during COVID before interest rates started rising. A high amount of labor mobility is an asymmetric process where leaving can occur in 2 weeks, while a position can take 3-6 months to replace. This leaves a lot of vacancies and job requisitions out in the market. Second, a lot of overhiring occurred during COVID, particularly for software engineers, as the world became much more digital because of remote work. Third, due to the high demand for software engineers, their pay and benefits became highly inflated, straining company cost structures and limiting their ability to hire more. Fourth, as interest rates started to rise, the ability for companies to obtain debt to fund operations/hiring was reduced and the overall economy weakened. This led to layoffs and greatly reduced labor mobility. When hiring slows, the number of job postings will naturally lower. While there is no doubt that AI will impact the economy, it was not the cause of the massive drop in demand of software development jobs.
Analyzing well
It's hard to analyze well - to be sure that your interpretation is sound and leads to a logical conclusion. It's even harder to prove causal relations. However, it is much easier to disprove causal relations. If an action happens before a trigger event, it's impossible for the trigger to cause the action. This is an easy way to disprove causal relations.
Setting up your question is critical. The right question can make a problem much easier to solve. It's extremely difficult to prove that something doesn't exist; It's much easier to prove that something does exist. As Nassim Taleb would say, "the evidence of a single black swan is enough to disprove the theory that black swans do not exist".
Don't fall for the hype that AI will cause the death of everything. Yes, the world will be impacted, but the future tends to be a lot more nuanced rather than white and black. People gravitate to the black and white because it is easier to understand, even if not true. Think through the actual implications and effects. A near term drop doesn't mean everything will disappear. Will the demand for software engineering jobs drop? Possibly. Will software engineering disappear? Absolutely not. Will the nature of software engineering change? Definitely. As always, there's always an XKCD for that:
