|Intelligence requires more than associating data with other data / Photo Credit: Fernando Cortes (via Shutterstock)|
Tristan Greene of The Next Web (TNW), a website and a series of annual conferences focused on new technology and start-up firms in Europe, noted that AI is still dumber than a baby even in 2020. However, AI and cognition expert and CEO of robotics company Robust AI Gary Marcus argued that we are just “scratching the surface of intelligence.” He asserted that Deep Learning won’t get us anywhere near human-level intelligence without Deep Understanding. Marcus wrote on The Gradient, a publication about AI, that GPT-2, a text generator, is good at parsing volumes of data while being bad at resembling a “basic human understanding of the information.” Like any other AI system, GPT-2 was unable to understand the words it has been trained on as well as the words it generates.
Type in a prompt and a Transformer neural network that has been trained on 43 gigabytes of data will manipulate 1.5 billion parameters, which will then generate new words. Due to the nature of GPT-2’s training, its sentences and paragraphs appear to be written by a fluent native speaker but it doesn’t understand those words. The generator doesn’t pick specific words, phrases, or sentences for their meaning because it just produces meaningless text that oftentimes appears grammatically correct. It can be used as a tool to inspire works of art, but as a source of information, GPT-2 has no value at all.
This means intelligence requires more than associating data with other data. Marcus stated that we will need Deep Learning to acquire Deep Understanding if we want the field to progress.
In a presentation Marcus gave at NEURIPS this year, he said that Deep Understanding requires AI to form a “'mental' model of a situation based on the information given.” He advocated for employing a hybrid approach, combining symbolic reasoning and other cognition methods with Deep Learning to make AI capable of deep understanding. This is better than spending billions of dollars in trying to fit more compute or larger parameter packages into outdated neural network architectures.