We study the non-conservation of the chiral charge of Dirac fields between past and future null infinity due to the Adler-Bell-Jackiw chiral anomaly. In previous investigations \cite{dR21}, we found that this charge fails to be conserved if electromagnetic sources in the bulk emit circularly polarized radiation. In this article, we unravel yet another contribution coming from the non-zero, infrared "soft" charges of the external, electromagnetic field. This new contribution can be interpreted as another manifestation of the ordinary memory effect produced by transitions between different infrared sectors of Maxwell theory, but now on test quantum fields rather than on test classical particles. In other words, a flux of electromagnetic waves can leave a memory on quantum fermion states in the form of a permanent, net helicity. We elaborate this idea in both $1+1$ and $3+1$ dimensions. We also show that, in sharp contrast, gravitational infrared charges do not contribute to the fermion chiral anomaly.
