Formal machine learning algorithms

Environment models

A set of time indices is a subset of real numbers. When $T$ is a set of non-negative integers $T \equiv \mathbb{N}$, the environment can be modeled as discrete time environment (chess, sequence of images). If $T \equiv [0,\infty)$, then $T$ is a continuous-time environment set of time indices (satellite control, speech recognition). We choose one of those based on suitability to specific modeling problem. Time represents the order in which events in the environment are experienced by the learning machine.

Let $\Omega_E \subseteq \mathcal{R}^e$ be a set of events. A function $\xi: T \to \Omega_E$ is called an event timeline function generated from $T$ and $\Omega_E$. $\xi(k)$ specifies a system state for time $k$. For a tic-tac-toe game it could be a 9-dimensional vector (X is 1, empty is 0, and O is -1):

$$\xi : \mathbb{N} \to \{-1,0,+1\}^9$$

For a machine that processes an auditory speech signal the function $\xi$ would look like this:

$$\xi: [0,\infty) \to \mathcal{R}$$

$\xi_{t_a,t_b}$ denotes the restriction of the event timeline function $\xi$ to the domain $\{t_a, t_a+1, t_a+2, \ldots, t_b-1\}$ when $T = \mathbb{N}$ and to the domain $[t_a, t_b)$ when $T = [0,\infty)$.