Relationships between learning and information compression based on PAC learning model

This paper is based on the concept of the learning function, which represents the input‐output relation of the learning algorithm. The learning process and the information compression process are formulated as the PAC learning function and the Occam function, respectively, and their equivalence is discussed. It is shown that the Occam function is always a consistent PAC learning function, while its converse is not always true. The weak Occam function which is obtained by weakening the condition concerning the information compression power of Occam function is defined anew and it is shown that the weak Occam function is always a consistent PAC learning function. Furthermore, a procedure is shown which derives the weak Occam function from the PAC learning function under a certain condition.