Spatiotemporal Program Learning
This project investigates how human adults, young children, and macaque monkeys learn abstract patterns from sparse data.
People learn languages, music, games, mathematics, and a seemingly limitless assortment of other structures across domains. How do we efficiently learn such a large variety of richly structured representations? To answer this question, we investigate structure learning mechanisms in human adults, children, and nonhuman primates using a highly unconstrained sequence prediction task.
One hypothesis is that people learn through program induction, synthesizing data-generating algorithms to explain what they observe. To instantiate this theory, we implemented a model that learns programs in a "Language of Thought" (LoT) with motor and geometry primitives. It predicts that people will (1) learn a large variety of programs from just a few datapoints, and (2) exhibit structured, multimodal uncertainty reflecting a distribution over programs that are consistent with the data.
The task revealed strikingly different inductive biases across groups. Adults and older (4-7 year-old) children show early multimodal uncertainty, before quickly converging on the true pattern for many richly structured sequencs.
In contrast, despite extensive training, two rhesus macaques succeeded mostly on linear and smoothly varying patterns, consistent with relying on local linear extrapolation. Three-year-olds' accuracy across patterns correlated with monkeys much more than with adults.
We compared multiple learning models to each group. While adults and children as young as four are best captured by sophisticated LoT program learning, monkeys and three-year-olds behave most like simple local linear extrapolation models.
Our results suggest that program induction is a powerful, early-emerging and perhaps distinctive mechanism of human structure learning. Open questions include how young children acquire and learn to flexibly deploy this mechanism, and what specific circumstances or capacities differentiate the development of program-learning abilities in human and nonhuman animals. For further discussion see associated papers!
Associated work: Preprint (2026) · NeurIPS paper (2023) · Data & code