{"id":94851,"date":"2024-06-06T16:12:22","date_gmt":"2024-06-06T16:12:22","guid":{"rendered":"https:\/\/www.globallogic.com\/?post_type=insightsection&p=94851"},"modified":"2025-01-24T12:05:05","modified_gmt":"2025-01-24T12:05:05","slug":"genai-and-the-meeting-of-the-minds","status":"publish","type":"insightsection","link":"https:\/\/www.globallogic.com\/insights\/blogs\/genai-and-the-meeting-of-the-minds\/","title":{"rendered":"GenAI and the Meeting of the Minds"},"content":{"rendered":"
At a recent Hitachi Energy conference, I saw a very interesting presentation by Hitachi partner nVidia\u2014the fabless semiconductor company whose GPUs are key drivers of the <\/span>GenAI revolution<\/span><\/a>. The speaker described nVidia not as a GPU company but rather as a \u201csimulation\u201d company. He described a spectrum of simulation technologies NVidia supports ranging from \u201cphysics-based\u201d to \u201cdata-based.\u201d<\/span><\/p>\n

As a person who was educated as a physicist, several light bulbs clicked on for me in this description. What the speaker meant, of course, was that simulations or video games can either be based on \u2018algorithms\u2019\u2014that is, a set of physical or un-physical laws (for fantasy worlds, for example)\u2014or they can use extrapolations based on data.<\/span><\/p>\n

When we as <\/span>developers write code<\/span><\/a>, we establish a set of \u2018laws\u2019 or rules for a computer to follow. Learned behavior, on the other hand, abstracts a set of patterns or probabilities from the data encountered. The latter is the nature of large language models\u2014they are not programmed; rather they are trained based on a selection of natural language text, photographs, music, or other sources of information.\u00a0<\/span><\/p>\n

The models essentially \u2018draw their own conclusions\u2019 in a learning process. (Or, more strictly speaking, the models are the artifacts embodying the learning that took place when an algorithm processed the training data.)<\/span><\/p>\n

Recommended reading: <\/i><\/b>Using AI to Maximize Business Potential: A Guide to Artificial Intelligence for Non-Technical Professionals<\/i><\/b><\/a><\/p>\n

Again, this stuck with me very forcefully as an analogy of the human learning process and of the way physics and science work.\u00a0<\/span><\/p>\n

There is a famous anecdote about the physicist Galileo, who was born in the 16th Century, observing the swaying of a chandelier during a church service in the town of Pisa Italy (of leaning tower fame). There was a breeze that occasionally set the chandeliers in motion with larger or smaller oscillations.\u00a0<\/span><\/p>\n

Galileo observed that regardless of how high the chandelier was blown by the wind, once it started to fall, a given chandelier always took the same amount of time to complete an oscillation. In other words, the time the chandelier took to swing back and forth depended only on the length of the chain holding it, not on the height when it was released.<\/span><\/p>\n

This is quite an extraordinary observation, and the fact that this phenomenon apparently was not noticed (or at least recorded and acted on) for the first 300,000 years or so of human history indicates the degree of insight and curiosity Galileo had.\u00a0<\/span><\/p>\n

Note that Galileo did not have a watch he could use to record the time\u2014they had not been invented yet, and could not have been until this \u2018pendulum effect\u2019 had been discovered. Galileo timed those initial oscillations using his pulse\u2014though he later refined his observations using, I presume, the water clocks or sand glasses that were known in his time.<\/span><\/p>\n

Why is this interesting? Because Galileo, like other discoverers, used observations or \u2018data\u2019 to infer patterns. From the data, he was able to make a prediction\u2014namely, that the period of a pendulum depends only on the length of the pendulum, and not on its height of oscillation, or (as was later found) its weight.<\/span><\/p>\n

Why is this important, and how does it relate to GenAI? There are two broad branches of Physics, called \u201cexperimental\u201d and \u201ctheoretical\u201d. The goal of experimental physics is to make observations and determine what happens. The goal of theoretical physics is to explain why something happens\u2014specifically, to discover the underlying principles that manifest themselves in observations, or that predict what will be observed.<\/span><\/p>\n

What is interesting to me in the context of GenAI is that there is a middle ground between these two areas of physics that is sometimes called phenomenology. The term phenomenology is used in different contexts, but back when I was a graduate student in high energy particle physics (theoretical physics, by the way) the word \u2018phenomenology\u2019 was used to describe predictions that we did not yet have the theory to explain.\u00a0<\/span><\/p>\n

In other words, we knew that something happened or would happen, but we didn\u2019t yet have a satisfactory explanation for \u201cwhy.\u201d<\/span><\/p>\n

Galileo, in his pendulum observations in the church and subsequently in his \u2018lab\u2019, was doing what today we would call experimental physics. That is, he was making observations about what happened, and describing what he saw.\u00a0<\/span><\/p>\n

In my limited historical research, I didn\u2019t find a record that he did so, but we can imagine that Galileo could have taken his observations one step further and made quantitative predictions about the behavior of pendulums. That is, based on his experimental results, he could have discovered that for small oscillations, the period of a pendulum was proportional to the square root of the pendulum’s length.\u00a0<\/span><\/p>\n

However, even if he had produced such a quantitatively accurate predictive model, history does not record that Galileo ever really understood WHY the pendulum rule he discovered was true. A satisfying qualitative explanation had to wait for roughly 100 years for Dutch scientist Christiaan Huygens\u2019 work on harmonic motion in 1673. A full quantitative explanation required Sir Isaac Newton to first invent calculus and lay out his three laws of motion. (For the theoretical basis of simple harmonic motion, such as a pendulum, <\/span>see here for example<\/span><\/a>.)<\/span><\/p>\n

So how does this history relate to GenAI?\u00a0<\/span><\/h2>\n

We can readily imagine our current-generation GenAI models acting like Galileo\u2014observing what happens, identifying patterns, and making extrapolations and predictions based on those patterns. We can even imagine them doing the curve fitting and other math required to turn those fresh observations into mathematical models.\u00a0<\/span><\/p>\n

It\u2019s more difficult to imagine a current-generation GenAI model acting like a Huygens or a Newton and inferring from first principles WHY something happens unless the model already contains that information and simply retrieves it.\u00a0<\/span><\/p>\n

I don\u2019t believe reasoning from first principles is impossible for GenAI, and people are working hard on enabling it. Approaches such as \u201cchain of thought\u201d and \u201ctrain of thought\u201d come close. But \u2018theory\u2019 is not the strong suit of current-generation (2024) GenAI technology. Current LLMs are \u201cphenomenologists\u201d, not \u201ctheorists\u201d, which is in no way intended to underrate their value.<\/span><\/p>\n

Why do we care about the theory? If we can predict \u201cwhat\u201d will happen, do we really care \u201cwhy\u201d?<\/span><\/p>\n

This is a good question, and it rapidly gets metaphysical, hinging on the nature of consciousness. Moreover, what constitutes a \u201csatisfying explanation\u201d and \u201cfirst principles\u201d gets really philosophical fast. But in a practical sense, we can see that both theory and phenomenology have value, each in a different context.<\/span><\/p>\n

Phenomenology has \u2018rough and ready\u2019 practical value. Astronomers and, earlier, astrologers could predict the phase of the moon and the progression of the seasons long before they understood that the Earth orbits the Sun, and the Moon orbits the Earth. These purely phenomenologically-based predictions had a profound impact on human history, including the invention of agriculture which, in turn, led to the creation of cities and civilization.\u00a0<\/span><\/p>\n

But it is the nature of the human mind to try to discern the reasons behind what it observes. People developed theories\u2014initially what we\u2019d now term religious or mythological\u2014to explain why the Sun and Moon behave as they do. They did this many centuries before the discovery of calculus and the law of gravity by Newton; the increasingly refined observations made by Kepler and, earlier, Galileo; and Copernicus\u2019 hypothesis that the earth obits the Sun. It is in the nature of humans to keep asking \u201cwhy\u201d until a satisfying \u2018theory\u2019 is presented to explain the observations.<\/span><\/p>\n

Watch: Getting GenAI Ready with GlobalLogic<\/i><\/b><\/a><\/p>\n

Besides being intellectually satisfying to us humans, the value of theory is that, by reducing observed behavior to an outcome of basic principles, it lets us solve problems and see connections that phenomenology alone does not.\u00a0<\/span><\/p>\n

For example, the theory of simple harmonic motion outlined in the Feynman lecture above not only explains the motion of pendulums (Galileo\u2019s observations), but also the vibration of plucked strings on musical instruments and the movement of weights on springs. When we generalize this slightly, driven harmonic motion (a pendulum pushed by the wind or by the escapement mechanism of a clock) also leads to insights in the area of \u201cresonance\u201d.\u00a0<\/span><\/p>\n

This, in turn, helps us understand diverse phenomena such as the structure of Saturn’s rings and the behavior of physical structures like bridges under the influence of an external force, such as the wind.\u00a0<\/span><\/p>\n

By uniting our understanding of multiple observations, a theory helps us discover the underlying connection between phenomena that initially appeared distinct. This process of forming a theory is not confined to physics but is something all of us do in everyday life. We have a theory of the motivations behind our spouse\u2019s or friend\u2019s behavior; as infants, we form the theory that an object continues to exist even when we don\u2019t see it; as students or engineers we form a theory of what it takes to get a good grade or promotion.\u00a0<\/span><\/p>\n

We also form \u2018theories\u2019 every day in the software space, when we develop an \u201carchitecture\u201d or algorithm that produces a (hopefully) simple system that solves not just one but multiple problems.\u00a0<\/span><\/p>\n

We also abstract out commonalities between diverse systems\u2014for example, logging, observability, and security\u2014and structure them as \u201ccross-cutting concerns\u201d rather than re-inventing them afresh for every system. In general, people consistently synthesize observations and try to discern the underlying cause behind them. It\u2019s our nature.<\/span><\/p>\n

The human brain functions using a combination of observation, phenomenologically-based prediction, and abstraction or \u201ctheory\u201d to understand what it observes and expects. Currently (in 2024), GenAI is strongest in the first two aspects\u2014observation and phenomenologically-based prediction.\u00a0<\/span><\/p>\n

To deliver on the \u2018holy\u2019 (or \u2018unholy\u2019) grail of <\/span>artificial general intelligence<\/span><\/a>, AI-based systems need to not only predict but also be able to form abstractions and \u2018theories\u2019 based on their observations and predictions. They will need to combine a \u2018Galileo brain\u2019 with a \u2018Sir Isaac Newton\u2019 brain.\u00a0<\/span><\/p>\n

I expect that we will indeed see such a \u2018meeting of minds\u2019 in GenAI, even though we\u2019re not fully there today. We have ourselves as examples that these two modes of thought can co-exist in a single entity. We also know first-hand the power of intelligence that not only predicts \u201cwhat,\u201d but also understands \u201cwhy.\u201d<\/span><\/p>\n

You might also enjoy:<\/span><\/i><\/p>\n