These days, news coverage on Artificial Intelligence (AI) conveys a simple equation: AI = Deep Learning. No wonder, billions of dollars revolve around this technology – and its marketing. Waves of enthusiasm shiver through the media when Google’s DeepMind recognizes a cat. Awe overcomes the tech masses when a deep learning knight with impressive pedigree enters the AI arena wearing Facebook’s standard. Mouths drop when reading about the $1 billion funding from Elon Musk & Co to promote OpenAI, an independent research lab – open only to deep learning advocates, it seems.
There is no doubt that neural networks have delivered great results in several fields of AI, like image and speech recognition (and still do; and probably still will). But the question I would like to pose is: are deep learning-based systems sustainable? Not only do they rely on enormous data sets to work, they also require a disproportionately high level of manual intervention (think annotations and fine-tuning of parameters). Not such a big problem for data champions like Google and Facebook who sit on billions of indexed web pages and social media posts. They have the petabytes of data and they have the budget to build computers powerful enough to process them.
As the swarm of Big Data keeps swelling, this approach means a race for more neural networks, more processors, more computing power. Seriously, in a world where, every day, 500 million tweets are produced, 205 billion emails are sent and received, 1,000 new articles are added to the English Wikipedia, is it the right path to engage in a race for CPUs? Is it our vision for the future to populate artificial cities with blinking, energy-devouring, giant processors, doomed to analyze mankind’s communication, in a desperate attempt to keep up with the pace?
I think our green and blue planet deserves a more elegant, sustainable solution, a solution that breaks the vicious circle and moves this boyish “mine is bigger” competition to where it belongs: to the archives of AI research.
According to Jeff Hawkins, the solution might lurk in the very way our brain processes information. Gary Marcus, a professor of psychology at New York University, follows a biologically inspired path too. Fact is: our brain does not need to see millions of cat pictures before recognizing one. A 4-year old child knows the difference between Jaguar the car and Jaguar the big cat, without having ingested tons of literature about cars and felines before-hand. Why? Because she heard her parents mention the word jaguar in the two different contexts and her brain automatically created the corresponding semantic spaces. When she hears that her father’s Jaguar has broken down, her brain immediately makes the analogy “broken down” = “machine” = “car” to disambiguate the meaning of the word. This cognition process based on similarity is one of the mechanisms that allow our brain to be the best performing computer we know of.
Now imagine an intelligent system that mimics that similarity-based approach. Instead of being fed colossal amounts of (annotated) data, it just needs to ingest a few examples – without any human supervision. Instead of performing an exhaustive screening of all potential solutions, it simply compares the similarity of the input with its knowledge base – just by measuring the semantic overlap. Instead of computing large and complex data structures, it processes generic binary vectors. Instead of being limited to a handful of semantic dimensions, it can apply many thousands of semantic features. Not only does such a biologically inspired system make execution on current CPUs orders of magnitude faster, it also dramatically improves the robustness and reliability of the results (see my previous post The false positives syndrome).
A community that is thoroughly entangled with a deep learning vision of AI will have difficulties admitting it, but here is the simple truth: Brute force is not the answer. Intelligence is.
