A survey detection channel overrides the pixels in an astronomical foundation model, and biases tomographic mean redshifts

AION‑1, a 39‑modality transformer built for astronomical surveys and trained on more than 200 million objects, exhibits a severe systematic bias when its survey segmentation map input is altered while keeping the image tokens unchanged. Causal interventions that replace the detection channel with a byte‑identical version cause the model’s predictions for flux, size, ellipticity and redshift to shift by factors ranging from 110 to 4 400 compared with a matched placebo, indicating that the presence of a detection gate at the field centre (correlation r = 0.47) drives the effect rather than the actual light enclosed by the mask (r = 0.30). Across 322 real blended sources the model disregards how the pipeline partitions light (R = ‑0.006), and providing contradictory catalogue photometry makes the model nine times worse than supplying no metadata at all.

The root of the bias lies in the incompleteness of the Legacy Survey pipeline, which fails to assign a segmentation mask to 3.68 % of targets. Propagating this miss rate through the model shifts tomographic mean redshifts by a median of 0.71 times the LSST Dark Energy Science Collaboration (DESC) requirement across 40 redshift assignments, exceeding the requirement in 12 cases; the worst redshift bin is affected by a factor of 8.3. The bias persists whether the missed targets are drawn uniformly or according to their measured magnitude dependence, and it grows with model scale. Spectroscopic information eliminates the effect, and simply withholding the detection channel removes the bias without measurable loss of performance.

Additional technical limits compound the problem. The model’s tokeniser encodes image patches into only 28 effective states versus 934 states for the spectral codec, and redshift outputs are constrained by quantisation. Sparse dictionary approaches prove unreliable as causal tools, achieving recovery rates of only 26‑75 % across 15 trials and moving up to 18 points on the seed alone. These findings highlight that detection‑channel reliance can undermine the scientific utility of foundation models in astronomy, urging developers to reconsider input architectures and to incorporate more robust, complete catalogues or spectroscopic data to mitigate systematic redshift errors.

⚡ Effects Interpreter