OrbitalAssets

Active Predictive Coding (ActPC) comprises a biologically inspired framework for learning and inference. By minimizing prediction errors between internal models and observed data, ActPC iteratively refines both latent representations and model parameters. Unlike backpropagation-based neural networks, ActPC emphasizes local error signals, making it inherently more suited for real-time, online learning and enabling integration of reinforcement learning and symbolic reasoning. Traditional gradient-based optimization in neural networks struggles to support large-scale real-time learning dynamics due to the brittleness of the underlying backpropagation algorithm, which requires carefully coordinated and synchronized updating across a large network (leading to a reliance on large batch-based updates), and which suffers convergence problems when neural architectures are too recurrent or otherwise too complex. These shortcomings lead to an unfortunate sociotechnical dynamic in which neural architectures oriented toward robust AGI-oriented learning, reasoning, and memory are insufficiently pursued because they tend to involve network topologies for which backpropagation will not easily converge. ActPC resolves these issues on both conceptual and mathematical levels, but can also suffer from lengthy convergence times and undesirable transient dynamics. Information-geometrically enhanced ActPC (ActPC-Geom) provides a compelling potential alternative: by incorporating measure-dependent operators derived from the Wasserstein distance, one aligns parameter updates with the natural structure of the probability distributions underlying ActPC, thereby accelerating learning and smoothing out digressive transient dynamics that might otherwise occur. Like most practical applications of information geometry, ActPC-Geom faces significant computational challenges. However, we believe these can be addressed via appropriate deployment of machine learning and reasoning algorithms within the geometric modeling itself (using neural approximators and information-geometry-guided kernel-PCA-based embeddings). Further, these ML and MR algorithms can be used to inject valuable cognitive properties into the neural learning process, alongside providing acceleration. Read the full paper to learn more: arxiv.org/abs/2501.04832