Feed of latest updates from the Neural Reckoning group https://neural-reckoning.org Science https://neural-reckoning.org/nr-logo.png https://neural-reckoning.org en-gb https://neural-reckoning.org/pub_factorization_genomic.html https://neural-reckoning.org/pub_factorization_genomic.html Wed, 12 Aug 2026 00:00:00 +0000 The genomic mechanisms that efficiently encode the initial architecture and synaptic connectivity of neural circuits remain poorly understood. We hypothesise that two primary mechanisms — spatial encoding and factorisation — enable a limited genome to initialise networks of billions of neurons. Spatial encoding, a form of indirect representation, compresses neural network parameters while enforcing structural continuity. Complementarily, we introduce a factorisation mechanism inspired by reaction-diffusion models, comprising a spatially invariant reaction rule and spatially variant diffusion dynamics. Based on this insight, we can efficiently abstract a neural network layer as a spatially invariant weight kernel (or filter) and its spatially variant transformations along the spatial dimensions. Thus, coupling this variant-invariant decomposition with spatial encodings can substantially reduce the size of the solution space explored by the genome. In addition, we show that this coupling leads to efficient itialisation of cortical maps, such as V1 orientation maps, and neural networks with the ability to generalise. In summary, coupling factorisation and spatial encodings can offer functional advantages to the evolving genome. https://neural-reckoning.org/pub_partial_recurrence.html https://neural-reckoning.org/pub_partial_recurrence.html Fri, 07 Aug 2026 00:00:00 +0000 Neural circuits are sparse and bidirectional. Meaning that signals flow from early sensory areas to later regions and back. Yet, between connected areas there exist some but not all pathways. How does this structure, somewhere between feedforward and fully recurrent, shape circuit function? To address this question, we designed a recurrent neural network model in which a set of weight matrices (i.e. pathways) can be combined to generate every network structure between feedforward and fully recurrent. We term these architectures partially recurrent neural networks (pRNNs). We trained over 25,000 pRNNs on a novel set of reinforcement learning tasks, designed to mimic multisensory navigation, and compared their performance across multiple functional metrics. Our findings reveal three key insights. First, in dense-cue environments, most pRNN architectures match or exceed the task performance, learning speed or robustness of fully recurrent networks, despite using as few as one quarter the number of parameters; in sparse-cue environments, many match but a substantial fraction underperform. These results demonstrate that partial recurrence can enable energy efficient, yet performant solutions. Second, each pathway's functional impact is both task and circuit dependent. For instance, feedback connections enhance robustness to noise in some, but not all contexts. Third, different pRNN architectures learn solutions with distinct input sensitivities and memory dynamics, and these computational traits help to explain their functional capabilities. Overall, our results demonstrate that partial recurrence can enable robust and efficient computation - a finding that may help to explain why neural circuits are sparse and bidirectional, and shows how these principles can inform the design of artificial systems. https://neural-reckoning.org/talk_2026_fens_chen_summer_school_learning_with_spikes.html https://neural-reckoning.org/talk_2026_fens_chen_summer_school_learning_with_spikes.html Wed, 15 Jul 2026 00:00:00 +0000 This was a lecture for a wide range of neuroscience students. I discuss why we should care about spiking neural networks, how to code and train them. I also talk about two studies that use the methods I've described, on neural heterogeneity and on the computational role of neuromodulation. https://neural-reckoning.org/pub_emergent_generalization.html https://neural-reckoning.org/pub_emergent_generalization.html Tue, 14 Jul 2026 00:00:00 +0000 Dimensionality reduction has proven powerful for identifying neural manifolds, which are low-dimensional structures underlying high-dimensional neural activity. These low-dimensional representations have improved the interpretability of population-level coding. Yet whether such low-dimensional representations are biologically relevant and confer functional advantages in learning systems, or merely reflect neuron-level activity, remains contested in neuroscience. We show that an explicit information bottleneck forcing a recurrent neural network to learn a low-dimensional representation is necessary for rotational and out-of-distribution generalisation in a time-series prediction task. Using information-theoretic measures of causal emergence, we characterise the dynamics of this representation across the memorisation-to-generalisation transition, finding a non-monotonic trajectory which shows an initial decrease, a minimum, and a subsequent rise to a maximum, even as prediction loss falls monotonically. This trajectory scales with task complexity, and the magnitude of emergent structure reliably predicts generalisation performance. Analysis of CA1 hippocampal activity in mice learning an alternating maze task reveals analogous non-monotonic emergence dynamics that track behavioural performance. Together, these findings indicate that the ability of neural networks to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition. https://neural-reckoning.org/pub_neuromodulation_enhances_sensory.html https://neural-reckoning.org/pub_neuromodulation_enhances_sensory.html Mon, 13 Jul 2026 00:00:00 +0000 Spiking neurons underlie the brain's extreme energy efficiency, and therefore have great potential in neuromorphic computing, although realising this efficiency in practice has proven challenging. We use neuromodulation, a biological mechanism that lets the network dynamically and contextually modify its own parameters. We find it substantially increases performance across a range of sensory processing tasks, including a challenging new speech-in-noise dataset we introduce, with very few additional resources (neurons, energy, parameters). Neuromodulatory networks are space and energy efficient thanks to two mechanisms that are directly relevant to neuromorphic computing and biology: firstly, they allow for an order-of-magnitude reduction in the number of neurons required; and secondly, they enable very sparse firing, achieving better results using orders of magnitude fewer spikes. Together, these properties may throw light on the computational role of neuromodulation in biology, and make neuromodulation an ideal mechanism to improve performance and efficiency for neuromorphic devices. https://neural-reckoning.org/pub_dual_memory_pathways.html https://neural-reckoning.org/pub_dual_memory_pathways.html Tue, 16 Jun 2026 00:00:00 +0000 Spiking neural networks excel at event-driven sensing. Yet, maintaining task-relevant context over long timescales both algorithmically and in hardware, while respecting both tight energy and memory budgets, remains a core challenge in the field. Here we address this challenge through an algorithm-hardware co-design effort. At the algorithm level, inspired by the cortical fast-slow organization in the brain, we introduce a neural network with an explicit slow memory pathway that, combined with fast spiking activity, enables a dual memory pathway architecture in which each layer maintains a compact low-dimensional state that summarizes recent activity and modulates spiking dynamics. This explicit memory stabilizes learning while preserving event-driven sparsity, achieving competitive accuracy on long-sequence benchmarks with 40-60% fewer parameters than equivalent state-of-the-art spiking neural networks. At the hardware level, we introduce a near-memory-compute architecture that fully leverages the advantages of the dual memory pathway architecture by retaining its compact shared state while optimizing data flow, across heterogeneous sparse-spike and dense-memory pathways. We show experimental results that demonstrate more than a fourfold increase in throughput and over a fivefold improvement in energy efficiency compared with state-of-the-art implementations. Together, these contributions demonstrate that biological principles can guide functional abstractions that are both algorithmically effective and hardware-efficient, establishing a scalable co-design framework for real-time neuromorphic computation and learning. https://neural-reckoning.org/talk_2026_ohbm_neuroai.html https://neural-reckoning.org/talk_2026_ohbm_neuroai.html Sat, 13 Jun 2026 00:00:00 +0000 Invited tutorial/research talk at the 2026 OHBM conference workshop on NeuroAI. I discuss the problem of the "garden of forking paths" in NeuroAI, multiverse analysis, and how I try to handle these issues. I talk about my work on modularity as an example of this in practice. https://neural-reckoning.org/pub_unifying_ds_graphs_mechanistic.html https://neural-reckoning.org/pub_unifying_ds_graphs_mechanistic.html Tue, 05 May 2026 00:00:00 +0000 Understanding how biological and artificial neural networks implement computation from connectivity is a central problem in neuroscience and machine learning. In neural systems, structural and functional connectivity are known to diverge, motivating approaches that move beyond direct connections alone. Here, we show that the spatial and temporal function of recurrent neural networks (RNNs) trained on hierarchically modular tasks can be recovered by modelling the network as a graph and analysing the multi-hop pathways between input and output units. In particular, decomposing these pathways by hop length reveals how the network temporally routes information. This perspective reframes regularisation: if function is implemented through multi-hop communication, then standard penalties such as L1 regularisation, which act only on individual weights, constrain single-hop structure rather than the multi-hop pathways that support computation. Motivated by this view, we introduce resolvent-RNNs (R-RNNs), which constrain multi-hop pathways and thereby induce temporal sparsity beyond that achieved by standard L1 regularisation. Compared with L1 regularisation, R-RNNs achieve improved performance by inducing temporal sparsity that matches the task structure, even when the task signal is sparse. Moreover, R-RNNs exhibit stronger sparsity-function alignment, reflected in their increased robustness under strong regularisation. Together, our results identify multi-hop communication as a key principle linking structure to function in recurrent networks, and suggest that sparsity should be defined over functional pathways rather than individual parameters. https://neural-reckoning.org/talk_2026_swc_will_we_understand_the_brain.html https://neural-reckoning.org/talk_2026_swc_will_we_understand_the_brain.html Thu, 23 Apr 2026 00:00:00 +0000 I identify four challenges for understanding the brain: (1) we take on problems that are too easy (2) our concepts don't scale (3) our understanding is unstable (4) doing theory and experiment together. This was the introduction to a panel discussion at the 2026 internal SWC symposium. https://neural-reckoning.org/pub_beyond_rate_coding.html https://neural-reckoning.org/pub_beyond_rate_coding.html Tue, 17 Feb 2026 00:00:00 +0000 The surrogate gradient descent algorithm enabled spiking neural networks to be trained to carry out challenging sensory processing tasks, an important step in understanding how spikes contribute to neural computations. However, it is unclear the extent to which these algorithms fully explore the space of possible spiking solutions to problems. We investigated whether spiking networks trained with surrogate gradient descent can learn to make use of information that is only encoded in the timing and not the rate of spikes. We constructed synthetic datasets with a range of types of spike timing information (interspike intervals, spatio-temporal spike patterns or polychrony, and coincidence codes). We find that surrogate gradient descent training can extract all of these types of information. In more realistic speech-based datasets, both timing and rate information is present. We therefore constructed variants of these datasets in which all rate information is removed, and find that surrogate gradient descent can still perform well. We tested all networks both with and without trainable axonal delays. We find that delays can give a significant increase in performance, particularly for more challenging tasks. To determine what types of spike timing information are being used by the networks trained on the speech-based tasks, we test these networks on time-reversed spikes which perturb spatio-temporal spike patterns but leave interspike intervals and coincidence information unchanged. We find that when axonal delays are not used, networks perform well under time reversal, whereas networks trained with delays perform poorly. This suggests that spiking neural networks with delays are better able to exploit temporal structure. To facilitate further studies of temporal coding, we have released our modified speech-based datasets. https://neural-reckoning.org/pub_bena_thesis.html https://neural-reckoning.org/pub_bena_thesis.html Thu, 01 Jan 2026 00:00:00 +0000 Modularity---the decomposition of complex systems into semi-autonomous, reusable parts---is widely regarded as a foundational organising principle of intelligence. Yet the concept remains elusive: definitions shift across fields, and the causal mechanisms linking structure to function are poorly understood. This thesis investigates modularity in neural networks from multiple angles, and in doing so, traces an arc from studying it as a measurable property to building self-organising substrates where it could emerge. We first disentangle modularity along two axes: its causal locus (physical vs. functional) and its causal role (imposed vs. emergent). Using this framework, we show that imposing structural modularity yields functional specialisation only under specific resource constraints and environmental separability. We then demonstrate that enforcing the metabolic costs of long-range connectivity via spatial embedding naturally produces sparse, modular topologies. These structurally modular networks outperform unstructured architectures on compositional tasks, proving physical wiring costs can drive functionally beneficial modularity. Pivoting to self-organisation, we demonstrate that locally-connected Neural Cellular Automata can master a wide range of computational primitives like matrix operations, and support neural network emulation. Next, we introduce a scale-free framework for self-organising digital circuits. Here, a topology-masked Transformer replaces global backpropagation with local message passing to act as a decentralised meta-optimizer. This system self-assembles functional Boolean circuits, maintains homeostasis, and dynamically re-routes logic around permanent hardware faults—exhibiting adaptive resilience that generalises from small to large circuits without retraining. Together, these investigations reveal modularity not as a single property to engineer, but as an emergent phenomenon---shaped by physical constraints, driven by functional demands, and mediated by local, self-organising rules. They suggest that the path toward robust, scalable intelligence may lie in cultivating substrates that grow, learn, and repair themselves from the ground up. https://neural-reckoning.org/talk_2025_mathematical_neuroscience_intelligence_as_resource_efficiency.html https://neural-reckoning.org/talk_2025_mathematical_neuroscience_intelligence_as_resource_efficiency.html Fri, 12 Dec 2025 00:00:00 +0000 I start by asking what could it mean to understand the brain. My answer is that we need to understand the nontrivial thing that only the brain can do ("intelligence", although that's a risky word). I argue for a definition of intelligence that includes resource efficiency as a core part, and suggest that this may be why machine learning approaches with such vast resources produce intelligence so unlike human intelligence. I illustrate with examples from multimodal perception and modularity, and discuss how this might be useful for our ongoing work in heterogeneity, delays and neuromodulation. https://neural-reckoning.org/pub_heterogeneous_delays_low_bit.html https://neural-reckoning.org/pub_heterogeneous_delays_low_bit.html Fri, 31 Oct 2025 00:00:00 +0000 Neural networks rely on learning synaptic weights. However, this overlooks other neural parameters that can also be learned and may be utilized by the brain. One such parameter is the delay: the brain exhibits complex temporal dynamics with heterogeneous delays, where signals are transmitted asynchronously between neurons. It has been theorized that this delay heterogeneity, rather than a cost to be minimized, can be exploited in embodied contexts where task-relevant information naturally sits contextually in the time domain. We test this hypothesis by training spiking neural networks to modify not only their weights but also their delays at different levels of precision. We find that delay heterogeneity enables state-of-the-art performance on temporally complex neuromorphic problems and can be achieved even when weights are extremely imprecise (1.58-bit ternary precision: just positive, negative, or absent). By enabling high performance with extremely low-precision weights, delay heterogeneity allows memory-efficient solutions that maintain state-of-the-art accuracy even when weights are compressed over an order of magnitude more aggressively than typically studied weight-only networks. We show how delays and time-constants adaptively trade-off, and reveal through ablation that task performance depends on task-appropriate delay distributions, with temporally-complex tasks requiring longer delays. Our results suggest temporal heterogeneity is an important principle for efficient computation, particularly when task-relevant information is temporal - as in the physical world - with implications for embodied intelligent systems and neuromorphic hardware. https://neural-reckoning.org/pub_learning_sound_loc_limited_sup.html https://neural-reckoning.org/pub_learning_sound_loc_limited_sup.html Fri, 10 Oct 2025 00:00:00 +0000 The acoustic cues used by humans and other animals to localise sounds are subtle, and change throughout our lifetime. This means that we need to constantly relearn or recalibrate our sound localisation circuit. This is often thought of as a “supervised” learning process where a “teacher” (for example, a parent, or your visual system) tells you whether or not you guessed the location correctly, and you use this information to update your localiser. However, there is not always an obvious teacher (for example in babies or blind people). Using computational models, we showed that approximate feedback from a simple innate circuit, such as that can distinguish left from right (e.g. the auditory orienting response), is sufficient to learn an accurate full-range sound localiser. Moreover, using this mechanism in addition to supervised learning can more robustly maintain the adaptive neural representation. We find several possible neural mechanisms that could underlie this type of learning, and hypothesise that multiple mechanisms may be present and provide examples in which these mechanisms can interact with each other. We conclude that when studying spatial hearing, we should not assume that the only source of learning is from the visual system or other supervisory signals. Further study of the proposed mechanisms could allow us to design better rehabilitation programmes to accelerate relearning/recalibration of spatial hearing. https://neural-reckoning.org/pub_spytorch_transmitter.html https://neural-reckoning.org/pub_spytorch_transmitter.html Wed, 17 Sep 2025 00:00:00 +0000 Friedemann Zenke’s 2019 paper, and its related coding tutorial SpyTorch, made it possible to apply modern machine learning to spiking neural networks. The innovation reinvigorated the field. https://neural-reckoning.org/pub_delays_ccn.html https://neural-reckoning.org/pub_delays_ccn.html Tue, 12 Aug 2025 00:00:00 +0000 Recent work has shown that the performance of spiking neural networks (SNNs) on temporally complex tasks improves significantly when axonal delays are treated as learnable parameters. This raises an important question: If temporal delays improve a network's computational capacity, how precise do synaptic weights need to be? In this work, we investigate the relationship between delay-based computation and weight precision by combining quantized synaptic weights with a range of learnable delays on a challenging neuromorphic audio task. Our results reveal that short delays contribute little to performance, whereas medium to long delays are critical. Building on this insight, we introduce a learnable thresholding mechanism to suppress short delays that can be effectively compensated for by weights. These findings suggest that delays can reduce the burden on weight precision, highlighting a promising direction for energy-efficient SNN design and offering new perspectives on the role of delay in biological and neuromorphic computation. https://neural-reckoning.org/pub_path_to_universal_nca.html https://neural-reckoning.org/pub_path_to_universal_nca.html Mon, 11 Aug 2025 00:00:00 +0000 Cellular automata have long been celebrated for their ability to generate complex behaviors from simple, local rules, with well-known discrete models like Conway's Game of Life proven capable of universal computation. Recent advancements have extended cellular automata into continuous domains, raising the question of whether these systems retain the capacity for universal computation. In parallel, neural cellular automata have emerged as a powerful paradigm where rules are learned via gradient descent rather than manually designed. This work explores the potential of neural cellular automata to develop a continuous Universal Cellular Automaton through training by gradient descent. We introduce a cellular automaton model, objective functions and training strategies to guide neural cellular automata toward universal computation in a continuous setting. Our experiments demonstrate the successful training of fundamental computational primitives - such as matrix multiplication and transposition - culminating in the emulation of a neural network solving the MNIST digit classification task directly within the cellular automata state. These results represent a foundational step toward realizing analog general-purpose computers, with implications for understanding universal computation in continuous dynamics and advancing the automated discovery of complex cellular automata behaviors via machine learning. https://neural-reckoning.org/pub_comob_soundloc.html https://neural-reckoning.org/pub_comob_soundloc.html Thu, 26 Jun 2025 00:00:00 +0000 Neuroscientists are increasingly initiating large-scale collaborations which bring together tens to hundreds of researchers. At this scale, such projects can tackle large-scale challenges and engage a wide range of participants. Inspired by projects in pure mathematics, we set out to test the feasibility of widening access to such projects even further, by running a massively collaborative project in computational neuroscience. The key difference, with prior neuroscientific efforts, being that our entire project (code, results, writing) was public from the outset, and that anyone could participate. To achieve this, we launched a public Git repository, with code for training spiking neural networks to solve a sound localisation task via surrogate gradient descent. We then invited anyone, anywhere to use this code as a springboard for exploring questions of interest to them, and encouraged participants to share their work both asynchronously through Git and synchronously at monthly online workshops. Our hope was that the resulting range of participants would allow us to make discoveries that a single team would have been unlikely to find. At a scientific level, our work investigated how a range of biologically-relevant parameters, from time delays to membrane time constants and levels of inhibition, could impact sound localisation in networks of spiking units. At a more macro-level, our project brought together 31 researchers from multiple countries, provided hands-on research experience to early career participants, and opportunities for supervision and teaching to later career participants. While our scientific results were not groundbreaking, our project demonstrates the potential for massively collaborative projects to transform neuroscience. https://neural-reckoning.org/pub_multisensory-channels-and-time.html https://neural-reckoning.org/pub_multisensory-channels-and-time.html Tue, 10 Jun 2025 00:00:00 +0000 Animals continuously combine information across sensory modalities and time, and use these combined signals to guide their behaviour. Picture a predator watching their prey sprint and screech through a field. To date, a range of multisensory algorithms have been proposed to model this process including linear and nonlinear fusion, which combine the inputs from multiple sensory channels via either a sum or nonlinear function. However, many multisensory algorithms treat successive observations independently, and so cannot leverage the temporal structure inherent to naturalistic stimuli. To investigate this, we introduce a novel multisensory task in which we provide the same number of task-relevant signals per trial but vary how this information is presented: from many short bursts to a few long sequences. We demonstrate that multisensory algorithms that treat different time steps as independent, perform sub-optimally on this task. However, simply augmenting these algorithms to integrate across sensory channels and short temporal windows allows them to perform surprisingly well, and comparably to fully recurrent neural networks. Overall, our work: highlights the benefits of fusing multisensory information across channels and time, shows that small increases in circuit/model complexity can lead to significant gains in performance, and provides a novel multisensory task for testing the relevance of this in biological systems. https://neural-reckoning.org/pub_fund_ecrs_not_moonshots.html https://neural-reckoning.org/pub_fund_ecrs_not_moonshots.html Mon, 26 May 2025 00:00:00 +0000 Large-scale projects run the risk of stifling scientific independence. Instead, let's explore alternative mechanisms of collaboration.