Abstract:

What is intelligence? In this talk, I will present my perspective: intelligence is positioning. In other words, we should always take the global view: a tree is never a single tree, but a tree positioned in a forest, which automatically encodes the relationships of this tree and other trees.

As the simplest example, I will present theoretical analysis of contrastive learning, showing that contrastive learning with the standard InfoNCE loss is equivalent to spectral clustering on the similarity graph. More importantly, in contrastive learning, the embedding of one object encodes the similarities to all other objects. I will then generalize this characterization to other foundation models, showing that all self-supervised learning methods are learning how to position objects into a presheaf category that automatically encodes pretext tasks. This generalization has three interesting theoretical implications. Finally, I will show that for foundation models, the positions of objects determines how model understand the relationships between them.