Precision Weighting
Restoring precision as an inferred belief state is claimed to resolve both iterative inference and depth degradation barriers within one architecture. In the linear case, precision-weighted prediction errors are formally equivalent to natu…
1 sources - 4 claims
Restoring precision as an inferred belief state is claimed to resolve both iterative inference and depth degradation barriers within one architecture. In the linear case, precision-weighted prediction errors are formally equivalent to natural-gradient descent. Discarding precision weighting makes predictive coding equivalent to homoscedastic noise with unweighted squared errors. The article identifies fixed identity precision as the shared simplification behind slowness and depth degradation in deep predictive coding.