Chapter 2: ‘Fitting’ a Data Set with a Function

2.1 Random Behavior

The Turning Point Test

What is random behavior? For the purposes of this dissertation, a list is called random if the list entries are completely non-deterministic; i.e., if the occurrence of any observation in the list in no way influences the occurrence of any other observation.

Should a list be random, then there is no sense in searching for deterministic components.

This section presents a test for random behavior, called the turning point test. The idea behind the test is this: if data is truly random, then certain behavior dictated by randomness is expected. A probabilistic comparison of properties of the actual data with what is expected—should the data be truly random—is used to support or deny the hypothesis of random behavior.

The turning point test is adapted from [Ken, 21-24]. Several other tests for randomness are discussed in the same cited section.

Some review of probability and statistics concepts is interspersed throughout this section. The reader is referred to [Dghty] for additional material.

A MATLAB program to apply the Turning Point Test is included in this section.

Turning points

The turning point test is so named because it counts the number of turning points in a finite list of data, and compares this number with what is expected, should the data be truly random.

For the turning point test that is developed in this section, it is assumed that the entries in a finite list are allowed to come from an interval of real numbers. Since there are an infinite number of choices for any list entry, the probability that two entries in the list are equal is zero. In particular, the probability that there are equal adjacent entries is zero.

If a finite list contains a large number of duplicate values, then the hypothesis that the list entries come from some interval is probably unwarranted, and the turning point test as developed here will not apply.

$\bigstar\,$ Probability considerations

An event $\,E\,$ is a subset of a sample space $\,S\,.$ The probability of $\,E\,$ is determined by how much ‘room’ $\,E\,$ takes up in $\,S\,.$

Suppose that two numbers are chosen from an interval $\,I\,.$ The corresponding sample space is $\,I\times I\,.$ The phrase ‘the probability that there are equal adjacent entries is zero’, means, precisely, that the measure of the set $\{(x,x)\,|\,x\in I\}\,$ (as a subset of $\,I \times I\,$) is zero.

Suppose that three numbers are chosen from an interval $\,I\,.$ The corresponding sample space is $\,I \times I \times I\,.$ The phrase ‘the probability that there are equal adjacent entries is zero’, means, precisely, that the set

$$ S := \{(x,x,y)\,|\,x,y\in I\} \cup \{(x,y,y)\,|\,x,y\in I\}\,, $$

(as a subset of $\,I \times I \times I\,$), has measure zero. The set $\,S\,$ is the intersection of two planes in $\,\Bbb R^3\,$ ($\,x = y\,$ and $\,y = z\,$) with the cube $\,I \times I \times I\,$; the resulting set is ‘thin’ in $\,\Bbb R^3\,.$

The hypothesis that the entries in a finite list come from an interval justifies taking the sample space, in the following development of the turning point test, to be all possible arrangements of three distinct values.

If, on the other hand, the entries come from some finite set, then the probability that there are equal adjacent entries is nonzero, and one would have to enlarge the sample space to account for these possible repeat values.

Let $\,\ell\,,$ $\,m\,,$ and $\,g\,$ be three distinct real numbers, with $\,\ell \lt m\lt g\,.$ The letters were chosen to remind the reader of this ordering: $\,\ell\,$ for ‘least’, $\,m\,$ for ‘middle’, and $\,g\,$ for ‘greatest’.

There are $\,3\cdot 2\cdot 1 = 6\,$ ways that these three numbers can be arranged in three slots. If the ordering is random, then these $\,6\,$ possible arrangements will occur with equal probability. Only four arrangements yield a turning point:

Thus, the probability of finding a turning point in a set of three distinct real values is $\,\frac46 = \frac 23\,.$

Let $\,S\,$ denote the sample space containing all possible arrangements of three distinct values, so that $\,S\,$ contains the six $3$-tuples investigated above.

Define a ‘counting’ random variable $\,C\,:\,S\rightarrow \{0,1\}\,$ via:

The probability that $\,b\,$ is a turning point is $\,\frac 23\,.$ It follows that the expected value of $\,C\,,$ denoted by both $\,E(C)\,$ and $\,\mu_C\,,$ is:

Here, $\,E\,$ is the expected value operator.

Define the function $\,C^2\,$ by:

Since $\,1^2 = 1\,$ and $\,0^2 = 0\,,$ it follows that:

Therefore, $\,E(C^2)\,$ also equals $\,\frac 23\,.$

Let $\,\text{var}(C)\,$ denote the variance of $\,C\,.$ Using the definition of variance, and the linearity of the expected value operator, one computes:

Consider now a list $\,\boldsymbol{\rm y} := (y_1,\ldots,y_n)\,$ of length $\,n\,.$ It is desired to count the number of turning points in this list. Since knowledge of both neighbors is required to classify a turning point, the first and last entries in a list cannot be turning points; so the maximum possible number of turning points present is $\,n - 2\,.$

Using the list $\,\boldsymbol{\rm y}\,,$ define $\,C_i\,,$ for $\,i = 2,\ldots,n-1\,,$ by:

Each random variable $\,C_i\,$ is distributed identically to the random variable $\,C\,.$

However, it is important to note that this collection of random variables $\,\{C_i\}_{i=2}^{n-1}\,$ is not a random sample corresponding to $\,C\,,$ because $\,C_i\,$ and $\,C_j\,$ are not independent for $\,0\lt |j-i|\le 2\,$; that is, when the $3$-tuples acted on by $\,C_i\,$ and $\,C_j\,$ overlap. This issue is addressed later on in this section.

Define a random variable $\,T\,$ by:

Then, $\,T\,$ gives the total number of turning points in the list.

Via linearity of the expected value operator:

Next, $\,\text{var}(T)\,,$ the variance of the random variable $\,T\,,$ is computed. Since

one first computes $\,E(T^2)\,$:

There are $ \,(n-2)(n-2) = n^2 - 4n + 4\,$ terms in the product $\,(C_2 + \cdots + C_{n-1})^2\,.$ It is necessary to count the number of terms of the form $\,C_iC_j\,$ for $\,j = i\,,$ $\,|j - i| = 1\,,$ $\,|j - i| = 2\,,$ and $\,|j - i| \gt 2\,$; that is, when the indices on $\,C\,$ are the same, or differ by exactly $\,1\,,$ exactly $\,2\,$, or more than $\,2\,.$

This ‘counting’ is easily accomplished by performing the multiplication as a matrix product, and analyzing the result:

The main diagonal has $\,n-2\,$ entries, each of the form $\,C_i^2\,.$ Thus, there are $\,n-2\,$ terms of the form $\,C_i^2\,.$

There are $\,(n-2)-1 = n-3\,$ entries on the first diagonal above and below the main diagonal, and these are the terms for which $\,|j - i| = 1\,.$ Thus, there are $\,2(n-3)\,$ terms of the form $\,C_iC_{i+1}\,.$

There are $\,(n-2)-2 = n-4\,$ entries on the second diagonal above and below the main diagonal, and these are the terms for which $\,|j-i| = 2\,.$ Thus, there are $\,2(n - 4)\,$ terms of the form $\,C_iC_{i+2}\,.$

The remaining terms are those for which $\,|j - i| \gt 2\,$; thus, there are

terms of the form $\,C_iC_j\,$ for $\,|j - i| \gt 2\,.$

With a slight abuse of summation notation, the findings thus far are summarized as:

In each sum, the index denotes the number of terms, and the argument depicts the form of the terms being added. The expectations of each term in (*) must be considered separately.

It has already been observed that $\,E(C_i^2) = \frac 23\,,$ since $\,C_i^2 = C_i\,.$

When $\,|j - i| \gt 2\,,$ the random variables $\,C_i\,$ and $\,C_j\,$ have non-overlapping domains. Under the assumption of randomly generated data, the occurrence or non-occurrence of a turning point for $\,C_i\,$ in no way influences the existence of a turning point for $\,C_j\,$ in this case; i.e., $\,C_i\,$ and $\,C_j\,$ are independent. Thus:

However, for $\,j\le 2\,,$ the random variables $\,C_i\,$ and $\,C_{i+j}\,$ have overlapping domains, and $\,E(C_iC_{i+j}) \ne E(C_i)\,E(C_{i+j})\,$; i.e., $\,C_i\,$ and $\,C_{i+j}\,$ are not independent for $\,j \le 2\,.$

The proof of this statement follows.

To evaluate $\,E(C_iC_{i+1})\,$ requires the investigation of existence of turning points in $\,4\,$ consecutive slots. For convenience of notation, let four distinct real numbers be labeled in order of increasing magnitude as $\,a\,,$ $\,b\,,$ $\,c\,$ and $\,d\,.$ There are $\,4\cdot 3\cdot 2\cdot 1 = 24\,$ ways that these four numbers can be arranged in four slots, as shown below:

For the arrangement $\,abcd\,,$ $\,C_i = 0\,$ and $\,C_{i+1} = 0\,.$ Thus, $\,C_iC_{i+1} = 0\,.$

For the arrangement $\,bacd\,,$ $\,C_i = 1\,$ (there is a trough). However, $\,C_{i+1} = 0\,$ (no turning point). Again, $\,C_iC_{i+1} = 0\,.$

For the arrangement $\,badc\,,$ $\,C_i = 1\,$ (there is a trough), and $\,C_{i+1} = 1\,$ (there is a peak). Thus, $\,C_iC_{i+1} = 1\,.$

Indeed, for the product random variable $\,C_iC_{i+1}\,$ to be nonzero, the arrangement of $\,a\,,$ $\,b\,,$ $\,c\,$ and $\,d\,$ must display both a trough and a peak. This occurs in $\,10\,$ of the $\,24\,$ possible arrangements, and hence:

Note that $\,\frac{5}{12}\ne \frac 23\cdot\frac 23\,,$ confirming that $\,C_i\,$ and $\,C_{i+1}\,$ are not independent.

The random variables $\,C_i\,$ and $\,C_{i+2}\,$ again have overlapping domains.

To evaluate $\,E(C_iC_{i+2})\,$ requires the investigation of turning points in $\,5\,$ consecutive slots. By methods similar to those just discussed, it can be shown that:

Substitution of the computed expectations into (*) gives:

Thus:

With both the mean and variance of the random variable $\,T\,$ now known, Chebyshev’s Inequality (stated next) can be used to compare the actual number of turning points from a given data set with the number that is expected under the hypothesis of random behavior.

This theorem states that for any random variable $\,X\,$ with mean $\,\mu\,$ and variance $\,\sigma^2\,,$ the probability that $\,X\,$ takes on a value which is at least distance $\,t\,$ from the mean, is at most $\,\frac{\sigma^2}{t^2}\,.$

Observe that Chebyshev’s Inequality is a ‘distribution-free’ result; that is, it is independent of the form of the probability density function for $\,X\,.$

It is interesting to note that no ‘tighter’ bound on $\,P(|X - \mu| \ge t)\,$ is possible, without additional information about the actual distribution of $\,X\,.$ That is, there exists a random variable for which equality is obtained in Chebyshev’s Inequality: $\,P(|X - \mu| \ge t) = \frac{\sigma^2}{t^2}\,$ (see, e.g., [Dghty, 123–124]).

Here is how Chebyshev’s Inequality and the turning point test are used to investigate the hypothesis that a given data set is random.

Suppose that a finite list of data values is given, where it is assumed that the entries in the list are allowed to come from some interval of real numbers.

As cautioned earlier, if there are a large number of identical adjacent values in the data set, then the hypothesis that the values come from some interval of real numbers is probably unwarranted, and the turning point test as developed here does not apply.

For any occasional adjacent data values that are identical, delete the repeated value, and decrease $\,n\,$ (the length of the list) by $\,1\,.$ For example, the data list

of length $\,14\,$ would be transformed to the list

of length $\,13\,,$ before applying the turning point test.

Let $\,N\,$ denote the length of the (possibly adjusted) data set.

Let $\,T_{\text{act}}\,$ denote the actual number of turning points in the (adjusted) list. Under the hypothesis of random behavior, the expected value and variance of the random variable $\,T\,$ that counts the number of turning points in the list are given by:

Let $\,d := |T_{\text{act}} - \mu|$ denote the distance between the actual and expected number of turning points. By Chebyshev’s Inequality, the probability that the distance of $\,d\,$ or greater between $\,\mu\,$ and $\,T\,$ would be observed, should the data be truly random, is:

If $\,\frac{\sigma^2}{d^2}\,$ is close to $\,0\,,$ then it is unlikely that $\,T_{\text{act}}\,$ turning points would be observed if the data were truly random. In this case, the hypothesis that the data is random would be rejected, and the search for deterministic components could begin.

If $\,\frac{\sigma^2}{d^2}\,$ is close to $\,1\,,$ then there is no reason to reject the hypothesis of random behavior. In this case, it may be fruitless to search for deterministic components.

Example 1: Applying the Turning Point Test

The data graphed below give the biweekly stock price of a mutual fund over a two-year time period.

There are no identical adjacent values. The total number of data points is $\,N = 61\,.$

The expected number of turning points, under the hypothesis of random behavior, is:

The actual number of turning points is

so that the distance between the actual and expected values is:

The variance of T is:

Chebyshev’s Inequality yields:

Thus, it is quite unlikely that only $\,28\,$ turning points would be observed, if the data were truly random. The hypothesis of random behavior is therefore rejected, and a search for deterministic components can begin.

Some data sets, as in the next example, are ‘locally’ random, and yet exhibit some deterministic behavior from a more ‘global’ point of view. In such instances, short-term data prediction may be unwarranted, whereas longer-term prediction may be possible.

For sufficiently large data sets, the turning point test can be used to help determine the ‘breadth of local random behavior’. This idea is explored in the next example.

Example 2

The data list graphed below was generated within MATLAB by first producing some pure data, via the MATLAB commands

i = [1:100];
y = i/6;

and then introducing noise by use of the MATLAB command rand(A).

The list  noisey  graphed below was generated by the MATLAB command:

noisey = y + 2*(rand(y) - 0.5);

The list  noisey  has $\,63\,$ turning points, so $\,T_{\text{act}} = 63\,.$

The list  noisey  has length $\,100\,,$ so $\,N = 100\,,$ and thus $\,\mu = \frac 23(100)\approx 66.67\,$ and $\,\sigma^2 = \frac{16(100)-29}{90} \approx 17.46\,.$

Then, $\,d = |66.67 - 63| = 3.67\,.$

Chebyshev’s Inequality yields:

Even though the data clearly illustrate a linear ‘trend’, there is no reason, based on this test, to reject the hypothesis of random behavior.

Here is what the turning point test is revealing in this situation: in moving through the list entry-by-entry, the numbers rise and fall in such a way that they could certainly have been produced by an entirely random process.

Indeed, since the slope of the ‘pure’ line is $\,\frac 16\,,$ and the noise is $\,\pm 1\,,$ it could take more than $\,6\,$ data entries before any increase due to the linear trend is observed.

Prediction of one data value into the future is unwarranted.

However, if one were to move through the list by taking, say, every fourth entry, then the ‘local’ random behavior may be overshadowed by the ‘global’ linear trend. This idea is investi- gated in a second, slightly different, application of the turning point test.

Example 2, continued

Generate a new list from noisey, by taking every fourth piece of data. Call the new list noisey4. This is accomplished via the MATLAB command

noisey4 = noisey(l:4:100);

Whereas the time list corresponding to  noisey  has spacing $\,T = 1\,,$ the time list corresponding to  noisey4  has spacing $\,T = 4\,.$ The new list  noisey4  is graphed below, and has length $\,N = 25\,.$

The list  noisey4  has $\,10\,$ turning points, so $\,T_{\text{act}} = 10\,.$

The list  noisey4  has length $\,25\,,$ so $\,N = 25\,,$ and thus $\mu = \frac 23(25)\approx 16.67\,$ and $\,\sigma^2 = \frac{16(25)-29}{90} \approx 4.12\,.$

Then, $\,d = |16.67 — 10| = 6.67\,.$

Chebyshev’s Inequality yields:

The hypothesis of random behavior is rejected for noisey4. Thus, predicting future values of the list noisey4, based on identified components, may be warranted. In other words, prediction of $\,4\,$ or more days into the future for the original list  noisey4  may be warranted.

In this example, the data clearly exhibit a linear trend. A method of ‘fitting’ data with a function of a specific form is discussed in Section 2.2.

In general, it is possible, with sufficient data, to continually produce sublists, xk, of a list x, by taking every $\,k^{\text{th}}\,$ entry from x. If the turning point test, when applied to xk, concludes that the hypothesis of random behavior is rejected, then prediction of $\,k\,$ or more units into the future for the original list  x  may be warranted.

MATLAB FUNCTION: Turning Point Test

The following MATLAB function is used by typing

y = tptest(x)

where:  x  is the INPUT row or column vector;  y  is the program OUTPUT.

The output matrix  y  consists of rows, where each row is of the form:

[length nofdup mu k TP P]

The variable  length  is the length of the list produced by taking every $\,k^{\text{th}}\,$ entry from x, and adjusting the resulting sublist to account for adjacent identical values. The number of duplicates found in the list is recorded in nofdup.

The variable  mu  is the expected number of turning points, if the behavior is truly random.

The variable  TP  is the actual number of turning points.

The variable  P  is $\,\frac{\sigma^2}{d^2}\,,$ from Chebyshev’s Inequality:

The test is repeatedly applied to sublists, until either the list is depleted, or until P $\lt 0.1\,.$

In Octave, the code looks like this:

The following diary of an actual MATLAB session shows the application of this Turning Point Test to the list  noisey  in Example 2.

Economics Application: Taking Advantage of Turning Points

If the hypothesis of random behavior is not rejected for given data, then it may be fruitless to seek deterministic components. However, the economics application discussed in this section shows how one can, even in this situation, often take advantage of the turning points (rises and falls) in stock market data.

The strategy is due to Eliason [Eli]. First, the underlying mathematical theory is presented. Then, an example illustrating the application of this theory to stock market trading is given.

The variable names used in the Martingale Algorithm are suggestive of common roles that these variables play in applications of the algorithm.

The function $\,A\,$ is the ‘Action’’ function; $\,W\,$ denotes a ‘Win’ and $\,L\,$ denotes a ‘Loss’.

The function $\,P\,$ is the ‘Profit’ function, and $\,B\,$ is the ‘Bet’ function. When a WIN occurs, the profit is increased by the previous bet; when a LOSS occurs, the profit is decreased by the previous bet.

Example 1: Applying the Martingale Algorithm

Let $\,\boldsymbol{\rm x} = (1,2,3,5)\,,$ and let $\,A(n) = (L, W, W, L, W,\ldots)\,.$ The table below summarizes the algorithm:

Observe that the algorithm STOPPED at $\,n = 5\,,$ and $\,P(5) = 11 = 1 + 2 + 3 +5\,$; that is, when the algorithm stopped, the value of $\,P\,$ is the sum of the digits in the initial list.

Example 2: Applying the Martingale Algorithm With a Different Action List

Suppose that a different action function is used:

The algorithm is summarized in the table below:

This time, the algorithm stopped at $\,n = 10\,,$ but again $\,P(n) = 11\,.$ The next theorem shows that this behavior is no coincidence:

Proof

The proof is by induction.

For $\,n \ge 1\,,$ let $\,T(n)\,$ be the statement:

‘If the algorithm stops at step $\,n\,,$ then $\,P(n) = M\,$’

A series of $\,W\,$’s in the corresponding action list is the quickest way to stop the algorithm. First, action lists containing only $\,W\,$’s are considered. Then, the induction step is applied to action lists that contain at least one $\,L\,.$

If $\,\boldsymbol{\rm x}(0)\,$ has only one entry, $\,\boldsymbol{\rm x}(0) = (x)\,,$ then $\,T(1)\,$ is true (see below).

If $\,\boldsymbol{\rm x}(0)\,$ has two entries, $\,\boldsymbol{\rm x}(0) = (x,y)\,,$ then $\,T(1)\,$ is true (see below).

If $\boldsymbol{\rm x}(0)\,$ has three entries, $\,\boldsymbol{\rm x}(0) = (x, y, z)\,,$ then it takes at least two steps to STOP the algorithm. In this case, $\,T(1)\,$ is vacuously true, and $\,T(2)\,$ is true (see below).

Now suppose $\boldsymbol{\rm x}(0)\,$ has $\,m\,$ entries, $\,\boldsymbol{\rm x}(0) = (x_1,\ldots,x_m)\,,$ where $\,m \ge 4\,.$

If $\,m\,$ is odd, then let $\,m = 2j-1\,$ for $\, j\ge 3\,.$ A series of $\,W\,$’s produces a STOP in $\,j\,$ steps, and $\,T(j)\,$ is true. For $\,1 \le k \lt j\,,$ $\,T(k)\,$ is vacuously true. Only the $\,W\,$’s are shown in the flow chart below.

If $\,m\,$ is even, then let $\,m = 2j\,$ for $\,j \ge 2\,.$ A series of $\,W\,$’s produces a STOP in $\,j\,$ steps, and $\,T(j)\,$ is true. For $\,1 \le k \lt j\,,$ $\,T(k)\,$ is vacuously true.

Suppose now that the action list has at least one $\,L\,.$ Suppose that $\,T(k)\,$ is true for all $\,k = 1,\ldots, N - 1\,,$ and consider the statement $\,T(N)\,$.

To motivate what follows, consider a typical flow chart that summarizes all possible actions on a list:

The following observations are important:

• Every STOP at step $\,N\,$ comes from a $\,W\,$ in step $\,N\,.$ Also, in these cases, $\,\boldsymbol{\rm x}(N — 1)\,$ has either $\,1\,$ or $\,2\,$ entries.
• Tracing back from a $\,W\,,$ there is a first $\,L\,,$ say in step $\,N - k\,,$ for $\,1 \le k \le N - 1\,.$
• Taking only $\,W\,$’s from step $\,N - k\,$ leads to a STOP that is earlier than step $\,N\,.$ The inductive hypothesis will be applied to this earlier STOP, to show that $\,T(N)\,$ is true.

Now, the induction argument. Suppose that a STOP occurs in step $\,N\,,$ and suppose that $\,\boldsymbol{\rm x}(N - 1)\,$ has one entry.

The flow chart below is useful in summarizing the results:

For ease of notation in what follows, the profit functions and lists are denoted by $\,P(n)\,$ and $\,\boldsymbol{\rm x}(n)\,,$ respectively, along the lower series of $\,W\,$’s, and are denoted by $\,P'(n)\,$ and $\,\boldsymbol{\rm x}'(n)\,$ around the TURN and along the upper series of $\,W\,$’s.

• Let $\,\boldsymbol{\rm x}(N - 1) = (x)\,.$ Since $\,P(N) = P(N - 1) + x\,,$ it follows that $\,P(N - 1) = P(N) - x\,.$
• Then, $\,\boldsymbol{\rm x}(N-2) = (x_1,x,x_2)\,$ for real numbers $\,x_1\,$ and $\,x_2\,.$ Since $$ P(N - 1) = P(N - 2) + (x_1 + x_2) \,, $$ it follows that: $$ \begin{align} P(N - 2) &= P(N - 1) - (x_1 + x_2)\cr &= P(N) - (x + x_1 + x_2) \end{align} $$
• Then, $\,\boldsymbol{\rm x}(N - 3) = (x_3,x_1, x, x_2, x_4)\,$ for real numbers $\,x_3\,$ and $\,x_4\,.$ Since $$ P(N - 2) = P(N - 3) + (x_3 + x_4) \,, $$ it follows that: $$ \begin{align} P(N - 3) &= P(N - 2) - (x_3 + x_4)\cr &= P(N) - (x + x_1 + x_2 + x_3 + x_4) \end{align} $$
• Continuing in this fashion, one has, in step $\,N - k\,,$ for $\,2 \le k \le N-1\,$: $$ \begin{align} &\boldsymbol{\rm x}(N — k)\cr &\quad = (x_{2k-3},x_{2k-5},\ldots,x_1,x,x_2,\ldots,x_{2k-4},x_{2k-2})\cr\cr \text{and}\cr\cr &P(N-k)\cr &\quad = P(N) - (x + x_1 + x_2 + \cdots + x_{2k-3} + x_{2k-2}) \end{align} $$

Taking the first $\,L\,$ that occurs in step $\,N — k\,,$ one has:

where $\,x_{2k-2}\,$ must equal $\,x_{2k-3}+x_{2k-4}\,,$ since the first and last entries of $\,\boldsymbol{\rm x}'(N-k-1)\,$ are summed and appended to $\,\boldsymbol{\rm x}'(N-k-1)\,$ to produce $\,\boldsymbol{\rm x}(N-k)\,.$

Since

it follows that: