Environment models
A set of time indices is a subset of real numbers. When is a set of non-negative integers , the environment can be modeled as discrete time environment (chess, sequence of images). If , then is a continuous-time environment set of time indices (satellite control, speech recognition). We choose one of those based on suitability to specific modeling problem. Time represents the order in which events in the environment are experienced by the learning machine.
Let be a set of events. A function is called an event timeline function generated from and . specifies a system state for time . For a tic-tac-toe game it could be a 9-dimensional vector (X is 1, empty is 0, and O is -1):
For a machine that processes an auditory speech signal the function would look like this:
denotes the restriction of the event timeline function to the domain when and to the domain when .