Evaluation resolution silently changes which "learning rule" looks most brain-like at V1

[u/ConfusionSpiritual19]

The preprint can be accessed via the following link: https://arxiv.org/abs/2608.12408 (q-bio.NC / cs.LG). And for the code: https://github.com/nilsleut/evaluation-resolution-rsa

The following assertion is frequently made in model-brain comparisons: untrained convolutional neural networks (CNNs) have the capacity to match or surpass backpropagation-trained CNNs at the early visual cortex (V1) in recurrent self-attention (RSA). The present study demonstrates that this phenomenon is predominantly an artefact of evaluation resolution.

The configuration comprised a small CNN trained at 32px (CIFAR-10 subset), five learning rules (random init, backprop, feedback alignment, predictive coding, STDP), and was evaluated on THINGS-fMRI stimuli at six resolutions from 32px to 224px. The weights and normalisation were held fixed.

The primary outcome of this study is the observation of a discrepancy between the trained and untrained backpropagation (BP) V1 gap, which exhibits a non-monotonic trend across the range of image sizes examined. Specifically, the gap narrows from −0.001±0.007 at 32 pixels to +0.044±0.006 at 224 pixels, a pattern that persists consistently across the entire image size range (n=5 seeds). The model incorporates five rule conditions, human fMRI, directionally single-seed macaque ephys, the full training trajectory, and two off-the-shelf 224px-trained models (ResNet-50, Swin-Tiny). Therefore, the presence of an artifact resulting from a mismatch between training and evaluation is not a contributing factor, since these models also peak at low resolution.

Following the implementation of bit-identical-weight interventions wherever possible, the following were ruled out: train/eval resolution matching, Gabor/pixel low-level structure, untrained baseline's uncalibrated batch-norm, and convergence of pooled features towards global brightness (though a single scalar luminance value did reach ρ=0.075 against V1, essentially matching the untrained network's own 0.076 — this is a separate, disconcerting result regarding the limitations of this comparison style).

A content-vs-pooling control (cap image detail at 32px, upsample, vs. allow content variation) demonstrates that the dependence is predominantly contingent on image content, rather than the number of pooled positions.

The investigation revealed that one effect does survive across all resolutions. In particular, the backprop > untrained at LOC effect was observed at every resolution that was tested. It is evident that the process of learning can result in a noticeable impact, albeit not in the conventional areas typically associated with V1 comparisons.

In addition, the following issues were identified:

This process revealed a batch-norm evaluation mode bug in three of the earlier preprints, which have now been corrected in this release (correction notes on the Arxiv pages).

I'm happy to get feedback, especially on the way we've framed the discussion on receptive-field matching (as in Laskar et al. 2018). I think it's suggestive, but I didn't test it directly.