The Encapsulation of Knowledge - Part 1
Exploring how models gain understanding
To acquire knowledge, one must study; but to acquire wisdom, one must observe.
-Marilyn vos Savant
There is an ongoing debate around whether AI and machine learning models contain knowledge. To those of us that use and understand these models, it's very clear that they encapsulate knowledge. That is to say, models create a structure and organization around the data that they used to learn that allow them to make inferences about the world. But, having knowledge is not the same as thinking, so let's set that aside for right now. What I think people are attempting to articulate in their statements about AI lacking knowledge is about the quality of knowledge that's represented. Let's walk through how these models store and structure knowledge.
We'll start with a basic example of a simple machine learning classifier. A classifier is a model that attempts to differentiate between two or more different classes. For example, between whether a team will win or lose, or if a picture contains a cat or dog. With lots of variables this can get complex pretty quickly. Let's simplify it and look at only two dimensions. Below is a plot of different classification algorithms against different data shapes or structures. To orient yourself:
Each row is for a different structure of data and each column is for a different machine learning classification algorithm. The first column contains the input data with no algorithm operating on it.
The input data is broken up into an equal number of points between two classes - red and blue. The first row is a swirl, the second row is a ring around a cluster, and the third row is two groups merging.
Each box is an intersection between a classification algorithm and a set of input data. The background colors in each plot are how each algorithm would classify a point in that location with a degree of confidence. Accordingly, dark red means highly confident that the point should be red and light blue means slightly confident that the point in that space should be blue.
If a point falls in the same color background, it has been classified correctly by the algorithm. If it falls on the opposite color background, the point has been misclassified.
The number in the bottom right of each plot is the classification accuracy.
Figure 1. A comparison of different classifiers on different datasets (click to expand). Courtesy of scikit-learn.
What do you observe? Different machine learning algorithms work in different ways. Each structures the same input data to a different view of the world. Some algorithms on average are better than others. Some algorithms are superb with certain types of data but struggle with other types of data. For instance, a linear SVM (third column) does best on the third row (93% accuracy) but worse than random on the second row (40% accuracy). A decision tree (sixth column) is best in class for the swirl (first row) but in the third tier for the ring around the cluster (second row). The way each algorithm structures their understanding of the data space differs, as can be seen in the shapes of the background colors.
Each algorithm structures knowledge differently, reflecting its own worldview. Structure of knowledge means the relationships in a decision space where models interpret a given piece of data. This is illustrated in the above plots by the background colors. It's important to understand that even when given the same data, different models have different decision spaces. This is similar to how individuals raised in similar environments have different views of the world. Some people make better decisions with the same pieces of information than others because they have a better structure of knowledge or understanding of how the world works. Likewise, some models perform better for the same input data.
All models encapsulate knowledge. However, that doesn't mean a model is any good at it. Models are judged by how well they perform on data they haven't seen before. Normally, we would assess the true performance of an algorithm by adding many new points into its decision space and evaluating its performance on the new points. Figuring out how to properly assess algorithms and models is a deep science. While assessing the quality of an assertion can get complex, you know intuitively what a bad assertion looks like. For instance if you extrapolate heavily from very little data points, you aren't really believable, like in the XKCD comic below.
Figure 2. Erroneous extrapolation. Courtesy of XKCD
Apart from the explanation above, what signs do we have that AI has a representation of knowledge? For one, it's being used to find things that haven't been found before. A team at the University of Liverpool used AI to discover a new material that rapidly conducts lithium ions for use in batteries. In a similar vein, AI was used to discover new antibiotics by a team at MIT. Only by having a good structure of knowledge about how things work is AI able to make these quality predictions.
People will judge AI models and point out the errors they make. Which is true, and we need methods to mitigate when errors occur in automated processes, however most people don't realize how fallible their own human knowledge is. Let me give you two well-known and major examples - bloodletting and earth's position in the solar system. Bloodletting was a "healing" process whereby doctors would drain blood from patients in an effort to make them overcome their illness. Obviously, we don't do this today, but it was used from early cultures until the late 19th century, during which time it was a fixture of human knowledge about how the body supposedly worked. People had a fundamental and erroneous belief that bloodletting helped people. The knowledge was bad.
Similarly, until Nicholas Copernicus (around the 1500s) almost all of human civilization believed that the Sun and everything else in space, rotated around the Earth. We've since proved this to be fundamentally false as well. People believed so confidently in both of these concepts that they structured life around these central tenants. Needless to say, their structure and understanding of the world had many fundamental errors. Similar to the the plot of the classifiers above, the models of thought used by humanity did not match up well to the data of how the universe worked.
Despite all this, human beings find ways to improve their knowledge. Likewise, advanced AI models have ways to effectively update their knowledge. People and AI models use a similar process to understand how to adjust their knowledge based on what happens in their contact to the world. A great mental framework of this comes from Tim Urban at Wait But Why. Tim's mental framework is about how to choose and secure goals while updating your beliefs about the world while interacting with it. At the same time, both you and the world are changing, which requires making dynamic adjustments to obtain a better understanding of the truth.
Figure 3. A mental framework for updating beliefs and goals. Courtesy of Wait But Why?
Both AI models and people interpret information, though they do so in different ways. The quality of a decision or action from a piece of information is based on how that person or model has structured their knowledge of the world and on what has been previously observed. What separates AI models from classical machine learning models is their ability to fully integrate large amounts of complex knowledge, in extremely high dimensional space with methods for continuous updating, while making high quality decisions. We know this because AI models are starting to mimic some human actions. Additionally, AI is discovering things that work in the real world that haven't been found before. These things can occur because the latest AI models have a high-quality understanding of how the world is structured. That's not to say AI is perfect by any means but the feats we see from these models would not be possible if their knowledge of how things worked was poor. Knowledge and understanding manifest in different ways, as we'll explore in part 2.