2.5　Creativity and artificial intelligence

Jürgen Schmidhuber's formal theory of creativity postulates that creativity，curiosity，and interestingness are by-products of a simple computational principle for measuring and optimizing learning progress.Consider an agent able to manipulate its environment and thus its own sensory inputs.The agent can use a black box optimization method such as reinforcement learning to learn（through informed trial and error）sequences of actions that maximize the expected sum of its future reward signals.There are extrinsic reward signals for achieving externally given goals，such as finding food when hungry.But Schmidhuber's objective function to be maximized also includes an additional，intrinsic term to model“wow-effects”.This non-standard term motivates purely creative behavior of the agent even when there are no external goals.A wow-effect is formally defined as follows.As the agent is creating and predicting and encoding the continually growing history of actions and sensory inputs，it keeps improving the predictor or encoder，which can be implemented as an artificial neural network or some other machine learning device that can exploit regularities in the data to improve its performance over time.The improvements can be measured precisely，by computing the difference in computational costs（storage size，number of required synapses，errors，time）needed to encode new observations before and after learning.This difference depends on the encoder's present subjective knowledge，which changes over time，but the theory formally takes this into account.The cost difference measures the strength of the present“wow-effect”due to sudden improvements in data compression or computational speed.It becomes an intrinsic reward signal for the action selector.The objective function thus motivates the action optimizer to create action sequences causing more wow-effects.Irregular，random data（or noise）do not permit any wow-effects or learning progress，and thus are“boring”by nature（providing no reward）.Already known and predictable regularities also are boring.Temporarily interesting are only the initially unknown，novel，regular patterns in both actions and observations.This motivates the agent to perform continual，open-ended，active，creative exploration.

According to Schmidhuber，his objective function explains the activities of scientists，artists，and comedians.For example，physicists are motivated to create experiments leading to observations obeying previously unpublished physical laws permitting better data compression.Likewise，composers receive intrinsic reward for creating non-arbitrary melodies with unexpected but regular harmonies that permit wow-effects through data compression improvements.Similarly，a comedian gets intrinsic reward for“inventing a novel joke with an unexpected punch line，related to the beginning of the story in an initially unexpected but quickly learnable way that also allows for better compression of the perceived data”.Schmidhuber argues that ongoing computer hardware advances will greatly scale up rudimentary artificial scientists and artists［clarification needed］based on simple implementations of the basic principle since 1990.He used the theory to create low-complexity art and an attractive human face.