TY - GEN
T1 - On Fisher Information Matrix for Simple Neural Networks with Softplus Activation
AU - Iida, Masazumi
AU - Takeishi, Yoshinari
AU - Takeuchi, Jun'ichi
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Fisher information of simple neural networks with the softplus activation function is argued. We show that, under certain conditions, FIMs of simple models have the similar interesting spectral structure as the one shown by Takeishi et al (2021) for networks with ReLU. This work helps us to understand why the FIM has such the structure.
AB - Fisher information of simple neural networks with the softplus activation function is argued. We show that, under certain conditions, FIMs of simple models have the similar interesting spectral structure as the one shown by Takeishi et al (2021) for networks with ReLU. This work helps us to understand why the FIM has such the structure.
UR - http://www.scopus.com/inward/record.url?scp=85136295077&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85136295077&partnerID=8YFLogxK
U2 - 10.1109/ISIT50566.2022.9834516
DO - 10.1109/ISIT50566.2022.9834516
M3 - Conference contribution
AN - SCOPUS:85136295077
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 3001
EP - 3006
BT - 2022 IEEE International Symposium on Information Theory, ISIT 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2022 IEEE International Symposium on Information Theory, ISIT 2022
Y2 - 26 June 2022 through 1 July 2022
ER -