In this paper innovation is discussed in the general context of learning. To this end some of the basic definitions of those terms recalled. Innovation can be understood as a novel way to solve a problem. Here the word "novel" is understood in a qualitative way: Any type of behavior can show a large variability and still be categorized into a discrete number of qualitatively different classes of behavior. For instance in general it is not difficult to discriminate "walking" from "running" as two classes of distinct types of behavior that each individually can have a vast multitude of variations. A technical innovation involves a qualitative different method of solving problems and is not just an improved way of performing a previously existing process. There are certainly examples where this distinction is not very clear-cut. The fact that qualitative changes take place on all scales is one of the characteristic features of complex adaptive system and is related to fractal structures and self-similarity.
The above definition of innovation can be applied at a multitude of levels and often involves the creation of new, specialized problems that need to be solved in order to improve the solution of a more general problem. For example the problem of producing extremely pure silicon crystals simply did not exist until it was recognized that this problem could be correlated to the age-old problem of improving the quality and speed of communication between humans over large distances. This example illustrates the multi-level complexity that is universally present for any innovative process.
The concept of learning as a persistent change of behavior is more general in the sense that it does not require novelty in the method of problem solving. In most cases learning will lead to a gradual improvement of the performance within the class of one existing strategy or behavioral pattern. It in claimed, however, that in any learning process some form of problem solving is involved and that performance of different behavioral patterns can be compared and measured with respect to a given task. In other words: Learning and innovation will not take place if there is not a payoff of some sort. This means that there exists a fitness function that assigns a single value to each behavioral pattern. This value measures its fitness in the context of the current environment.