Reviewing Predictions for 2024
Maintaining intellectual honesty
Maintaining intellectual honesty
All good machine learning practice starts with evaluating the performance of your predictions. Whether you're measuring accuracy or using another metric, the goal is the same: to understand how well your model is doing. But this isn't just about algorithms—it’s a principle that applies to your own thinking. Improving your mental models works much the same way. By reflecting on how your predictions and assumptions turn out, you gain insights to refine your thinking and make better judgments in the future. I made a series of predictions right before 2024 began. In this article, I’m revisiting them to assess how they held up. Here's what I predicted and how those predictions faired.
Emergence of sunbirds: I'm defining “sunbird” as a "reverse snowbird" - that is a person who travels to cooler places during the summer and returns to where he or she lives for the rest of the year. For instance, during summer months the heat, water issues, or electricity costs might be too much for someone in Phoenix or Houston. That person would instead travel to New Hampshire for the summer until things cool down.
I don't think this blew up in the way I thought it would but people did write about it.
Proliferation of AI agents: This coming year, I believe the public will move away from interacting with AI models directly and will interact more with AI agents. This is an interaction layer that will be tailored for a class of tasks, similar to other software. This will induce many different agents, which use different tools or are utilized for different tasks, to be created.
This happened. The conversation has shifted from using AI to using agents and agentic systems.
Emergence of tool marketplaces for agents: Agents work much better when they have access to tools as opposed to having to figure out how to solve every task from scratch each time. For this reason, I think we will see the emergence of marketplaces for tools that AI agents can access. They will probably be in the form of paid API calls that AI agents can orchestrate together to carry out a designated task. This might change in the future as AI methods improve.
This also happened. Here's a non-exhaustive list of various marketplaces:
Focus on output verification: This coming year there will be a big focus on verifying the outputs of AI models. The world was taken by storm with the promise of AI in 2023 and started to get a cursory understanding of where all the warts are as they delved into errors and "hallucinations". What I saw this year is that many major companies are reluctant to fully deploy AI models because they can't trust the reliability of the outputs. Companies want to; They salivate at the potential of AI. However, there's too much risk with unreliable outputs. Most generative AI models are used with heavy human handholding and guidance. To unlock the true power of AI and to use it at scale, automated verification is needed. In 2024, I see the main focus on AI being how to verify the outputs of AI models and if they are off, to prevent those outputs from being further used. This will be a countermovement to the argument "just trust the algorithm".
Big leaps were made in this area this year. There are multiple frameworks like Guardrails AI that can assist in verifying the outputs of models and re-prompt if necessary. The current focus on test-time compute also provides a way to embed more verifications into model outputs. Speed and correctness are now the two main concerns when implementing AI models.
AI will be blamed for something humans did: This coming year, I believe we'll see AI used as a scapegoat for things that go wrong. Once in the mindset of scapegoating AI, I believe there will be a major event where AI will be blamed but it later comes out the event was fully human driven.
I'll call this half right. While people have had a lot of issues in how they use and apply AI there was no huge scandal. The closest thing I heard of was AI being used as a scapegoat to drive layoffs but that was more of an industry wide trend than a single event. Let me know if you heard of an AI scapegoat event.
Emergence of voice ui/ux designers: This year, I think we'll see the emergence of individuals who's sole role is to figure out how to craft a voice experience for interacting with AI models and AI agents. They will craft personalities, back and forth sequences, personalized understanding of their users, and proactive actions to create the best experience for interacting with an AI model or agent. I'm not sure what they will be called, perhaps interaction designers or AI spirit makers or something completely novel.
While companies like Oracle and Soundhound are hiring voice interaction designers, this hasn’t taken off like I thought it would.
Rise of AI regulation cottage industry: It's no surprise that AI is swiftly coming under regulatory scrutiny around the world. As I've written before, AI is ultimately about power, and that threat causes regulation. In response, an entire cottage industry will emerge to help companies deal with AI regulation for the systems they deploy.
The EU passed their AI regulation act but it doesn’t go into full effect until 2026. While big consulting companies started providing services around AI regulation, the actual regulations have been slow to affect industry. There’s been a lot of talk with less action which made the rise of the AI regulation cottage
industry slow.
AI data fighting: Foundation models are data hungry, and large model creators are scouring for ways to get more data to feed their models. At the same time, foundation models are enabling the proliferation of content. I think that the creators of the foundation models will poison their own outputs to affect the training processes of other foundation models that scrape their data. That is, each foundation model will put out modifications to generated content that is invisible to a human but harms an "enemy" foundation model during the training process.
Artists started to deploy tools this year that made it difficult for generative AI models to train on their images. I haven't found any evidence of foundation models affecting each other but if you have, let me know.
Proliferation of autonomous weapons systems: I think 2024 will be the year when autonomous weapons systems come into the public spotlight. There's been a steady march towards these systems, but with recent advances, I believe we'll see autonomous weapons systems begin to take up more of the battlefield.
Many are calling the Ukraine war as the start of the robot wars due to the deployment of autonomous weapons systems. This has caught a lot of attention as the larger powers are watching to see how these weapons will change the way wars are fought.
Massive heat events: This past year was the most anomalous for heat events, as seen in the picture below. It was so anomalous that it has climate scientists freaked out (more than they already are) because it might mean ice in certain places isn't coming back. At the same time, climatologists predict a very strong El Niño in 2024. These combined factors make it appear that we will have massive heat events in 2024. This past summer in 2023 was hot, very hot. It appears that 2024 will be even hotter. The phrase that's stuck with me is '2023 was one of the hottest summers ever, but it will be the coolest summer you experience for the rest of your life'.
2024 was the hottest year on record. That trend is likely to continue. There were large heat wave events in the US and Mexico that resulted in the deaths of at least 1,161 people.
Massive political disinformation in the election cycle personalized through AI: This feels like a no-brainer to me, and I even hesitate to put it on this list for that reason. Elections always see candidates use the latest advances in mental manipulation to try and win seats of power. I think we'll see an onslaught of hyper-personalized political content and misinformation in the upcoming elections.
While there definitely were AI generated campaign ads, the impact was a bit less clear. Depending who you read or talk to, this either had a big impact on the election or it didn't. Wired had the best take, lots of unexpected things happened around AI usage and subsequently the ways AI was used were less detectable.
Slightly better than half correct, but it’s worth remembering that the significance of an impact often outweighs the frequency of success when evaluating predictions. Some of these outcomes may have surprised you, while others might have felt predictable. Either way, I hope they offered valuable perspective on the year. As I’ve said before, reflecting on predictions and holding ourselves accountable sharpens decision-making over time. I hope this encourages others to openly review and assess their own forecasts.