論文

Neural networks acquire internal representations through learning. In this work we formulate stochastic gradient descent (SGD) as a Markovian stochastic process and derive a Fisher-information flow speed limit that bounds the rate at which trainable parameters can acquire information about data-generating parameters. The resulting inequality decomposes the upper bound on the information flow into drift and noise contributions, thereby quantifying the roles of deterministic learning forces and SGD-induced fluctuations from an information-theoretic perspective. We verify the bound in analytically tractable basis-function linear regression, where the information budget predicted by the bound reproduces the ordering and characteristic time scales with which different data-generating parameters are encoded in the learned parameters. These results establish Fisher-information speed limits as a quantitative framework for understanding which properties of the data-generating process are acquired, in what order, and how rapidly they can be encoded in the trainable parameters.

We investigate the consequences of information exchange between a system and a measurement-feedback apparatus that cools the system below the environmental temperature. A quantitative relationship between entropy pumping and information acquired about the system is derived, showing that, independent of the concrete realization of the feedback, the latter exceeds the former by a positive amount of excess information flow. This excess information flow satisfies a trade-off relation with the precision of the feedback force, which places strong constraints on both the information-theoretic cost of feedback cooling and the required magnitude of the feedback force. From these constraints, a fundamental lower bound on the energetic cost of optical feedback cooling is derived. Finally, the results are demonstrated for feedback cooling by coherent light scattering. We show that measurement precision is the major factor determining the attainable temperature. Precise measurements can also be leveraged to reduce the required feedback force, leading to significantly more energy-efficient cooling close to the fundamental bound for realistic parameter values.