Abstract
Humans are experts at understanding what they see. Similarity and analogy play a significant role in making sense of the visual world by forming analogies to similar images encountered previously. Yet, while these acts of visual reasoning may be commonplace, the processes of visual analogy are not yet well understood.
In this dissertation, I investigate the utility of representing visual information in a fractal manner for computing visual similarity and analogy. In particular, I develop a computational technique of fractal reasoning for addressing problems of visual similarity and novelty. I illustrate the effectiveness of fractal reasoning on problems of visual similarity and analogy on the Raven’s Progressive Matrices and Miller’s Analogies tests of intelligence, problems of visual novelty and oddity on the Odd One Out test of intelligence, and problems of visual similarity and oddity on the Dehaene test of core geometric reasoning. I show that the performance of my computational model on these various tests is comparable to human performance.
Fractal reasoning provides a new method for computing answers to such problems. Specifically, I show that the choice of the level of abstraction of problem representation determines the degree to which an answer may be regarded as confident, and that that choice of abstraction may be controlled automatically by the algorithm as a means of seeking that confident answer. This emergence of ambiguity and its remedy via problem re-representation is afforded by the fractal representation. I also show how reasoning over sparse data (at coarse levels of abstraction) or homogeneous data (at finest levels of abstraction) could both drive the automatic exclusion of certain levels of abstraction, as well as provide a signal to shift the analogical reasoning from consideration of simple analogies (such as analogies between pairs of objects) to more complex analogies (such as analogies among triplets, or larger groups of objects).
My dissertation also explores fractal reasoning in perception, including some aspects of biologically-inspired imprinting and bistable perception. In particular, it provides a computational explanation of bistable perception in the famous Necker cube problem that is directly tied to the process of determining a confident interpretation via rerepresentation.
Thus, my research makes two primary contributions to AI theories of visual similarity and analogy. The first contribution is the Extended Analogy By Recall (ABR*) algorithm, the computational technique for visual reasoning that automatically adjusts fractal representations to an appropriate level of abstraction. The second contribution is the fractal representation itself, a knowledge representation that add the notions of self-similarity and re-representation to analogy making.
