Abstract
25 November 2015 – It is common scientific practice to investigate phenomena, which cannot be explained by an existing set of theories, scientifically by applying statistical methods. This is how medical research has led to coherent treatment procedures, which provided a great deal of usefulness to patients. By observing many cases of a disease and by identifying and accounting its various cause and effect relationships, the statistical evaluation of these records allowed to make thoughtful predictions and to find adequate treatments as countermeasures. Nevertheless, since the rise of molecular biology and genetics, we can observe how medical science moves from the time-consuming trial and error strategy to a much more efficient, deterministic procedure that is grounded on solid theories and will eventually lead to a fully personalized medicine.
The science of language had a very similar development. In the beginning, extensive statistics analyses led to a good analytical understanding of the nature and the functioning of human language and culminated in the discipline of linguistics. With the increasing involvement of computer science into the field of linguistics, it turned out that the observed linguistic rules were extremely hard to use for the computational interpretation of language. In order to allow computer systems to perform language based tasks comparable to humans, a computational theory of language was needed and as no such theory was available, research turned again towards a statistical approach by creating various computational language models derived from simple word count statistics. Although there were initial successes, statistical Natural Language Processing (NLP) suffers two main flaws: The achievable precision is always lower than the one of humans and the algorithmic frameworks are chronically inefficient.
The Semantic Folding Theory (SFT) is the attempt to develop an alternative computational theory for the processing of language data. While nearly all current methods of processing natural language based on its meaning use in some form or other word statistics, Semantic Folding uses a neuroscience rooted mechanism of distributional semantics. After capturing a given semantic universe of a reference set of documents by means of a fully unsupervised mechanism, the resulting semantic space is folded into each and every word-representation vector. These vectors are large, sparsely filled binary vectors. Every feature bit in this vector not only corresponds but also equals a specific semantic feature of the folded-in semantic space and is therefore semantically grounded. The resulting word-vectors are fully conforming to the requirements for valid word- SDRs (Sparse Distributed Representation) in the context of the Hierarchical Temporal Memory (HTM) theory by Jeff Hawkins. While the HTM theory focuses on the cortical mechanism for identifying, memorizing and predicting reoccurring sequences of SDR patterns, the Semantic Folding theory describes the encoding mechanism that converts semantic input data into a valid SDR format, directly usable by HTM networks.
The main advantage of using the SDR format is that it allows any data-items to be directly compared. In fact, it turns out that by applying Boolean operators and a similarity function, many Natural Language Processing operations can be implemented in a very elegant and efficient way.
Douglas R. Hofstadter’s Analogy as the Core of Cognition is a rich source for theoretical background on mental computation by analogy. In order to allow the brain to make sense of the world by identifying and applying analogies, all input data must be presented to the neo-cortex as a representation that is suited for the application of a distance measure.
The two faculties – making analogies and making predictions based on previous experiences – seem to be essential and could even be sufficient for the emergence of human-like intelligence.
