CSCE 235
Handout
12: Other Induction-Related Issues
March 3,
2002
Second
Principle of Mathematical Induction:
Strong Induction
With this form, we use
the same basis step as in the usual Principle of Mathematical Induction, but we
use a different inductive step. We
assume that 
is true for j =
1,…, k and show that 
must also be true
based on this assumption.
Basis Step: The proposition 
is shown to be true.
Inductive Step: It is shown that 
is true for every
positive integer k.
The
Well-Ordering Property
Every nonempty set of nonnegative integers has at least one element.
Why
is Mathematical Induction Valid?
The reason comes from
the well-ordering property. Suppose we
know that 
is true and that the proposition 
is true for all
positive integers k. To show
that 
must be true for all
positive integers, assume that there is at least one integer for which 
is false. Then the set S of positive integers
for which 
is false is
nonempty. Thus, by the well-ordering
property, S has at least one element, which will be denoted by m. We know that m cannot be 1, since 
is true. Since m is positive and greater than
1, m-1 is a positive integer.
Furthermore, since m-1 is less than m, it is not in
S. So 
must be true. Since the implication 
is also true, it must
be the case that 
is true. This contradicts the choice of m. Hence, 
must be true for
every positive integer n.
Recursive
Definitions
We use two steps to define a function with the set of nonnegative integers as its domain:
Basis step: Specify the value of the function at zero.
Recursive step: Give a rule for finding its value at an integer from its values at smaller integers.
Such a definition is called a recursive or inductive definition.
Structural
Induction
Instead of using mathematical induction directly to prove results about recursively defined sets, we can use a more convenient form of induction knows as structural induction. A proof by structural induction consists of two parts:
Basis step: Show that the result holds for all elements specified in the basis step of the recursive definition to be in the set.
Recursive step: Show that if the statement is true for each of the elements used to construct new elements in the recursive step of the definition, the result holds for these new elements.
The validity of
structural induction follows from the principle of mathematical induction for
the nonnegative integers. To see this,
let 
state that the claim
is true for all elements of the set that are generated by n or fewer
applications of the rules in the recursive step of a recursive definition. We will have established that the principle
of mathematical induction implies the principle of structural induction if we
can show that 
is true whenever n
is a positive integer. In the basis
step of a proof by structural induction, we show that 
is true. That is, we show that the result is true for
all elements specified to be in the set in the basis step of the
definition. A consequence of the
inductive step is that if we assume 
is true, it follows
that 
is true. When we have completed a proof using
structural induction, we have shown that 
is true and that 
implies 
. By mathematical
induction if follows that 
is true for all
nonnegative integers n.
An Application of
Mathematical Induction
What is the use of Induction? There are many ways in which Induction is useful. Here, we talk about one useful application (as suggested by Prof. Vinod Variyam). That is, we can use induction in the analysis of loop invariants.
What are loop invariants? First, what is a loop? In computer programming, we have for and while loops; in circuitry, we have closed circuits; in business or systems, we have feedback loops, quality control loops, product design loops, R&D loops; in control systems, we have adjustment loops to make sure the systems are within a targeted zone; and so on. In the following, we will use a while loop as an example. But keep in mind that there are loop invariants in other types of loops as well.
When we debug a loop, we usually check to see whether it is an infinite loop—i.e., whether it terminates. Sometimes, this is difficult to do when the loop is very complex! This is when loop invariant analysis can help.
There are several definitions for a loop invariant:
From http://www.cs.duke.edu/~ola/patterns/plopd/invariant.html:
A loop invariant is a Boolean expression that is true each time the
loop guard is evaluated. Typically the Boolean expression is composed of
variables used in the loop. The invariant is true every time the loop guard is
evaluated. In particular, the initial establishment of the truth of the
invariant helps determine the proper initial values of variables used in the
loop guard and body. In the loop body, some statements make the invariant
false, and other statements must then re-establish the invariant so that it is
true before the loop guard is evaluated again.
From http://www.cs.miami.edu/~burt/learning/Math120.1/Notes/LoopInvar.html:
A loop invariant is a formal statement about the relationship
between variables in your program which holds true just before the loop is ever
run (establishing the invariant) and is true again at the bottom of the loop,
each time through the loop (maintaining the invariant).
So, here is an example:
…
// make sure that the loop invariant is true
before the loop
while (test condition is true) {
…
…
// bottom of the loop
// make sure that the loop invariant is
true here
…
}
// termination + loop invariant = goal
Now, how can Induction be of help?
For example, let the loop’s objective be “Compute the factorial.” Now we have the following:
…
int i = 1;
int product_so_far = 1;
while (i <= N) { // test condition,
where N is the number to
// to compute the factorial for
product_so_far = i*product_so_far;
i = i + 1;
}
printf(“factorial of %d = %d\n”,n,product_so_far);
…
Here is the loop invariant: i and product_so_far. We set the invariant true before the loop by:
int i = 1;
int product_so_far = 1;
And then inside the loop, we update product_so_far and then increment i to keep the loop invariant true: product_so_far = i*product_so_far! That is, for all i, product_so_far[i] = i*product_so_far[i-1]. The termination condition is when i > N.
And when we get out of the loop, we must have met the termination condition and the loop invariant must be true. And thus, we have our goal: the factorial of N.
So, what does this have to do with Induction?
Well, the initial setup of the loop invariant before the while loop is the setup of the Basis for Induction. The handling of the loop invariant at the bottom of the while loop is the Induction step. In the basis, we say that product_so_far is 1 when i is 1. In the induction step, we say that if product_so_far[i] = i*product_so_far[i-1], then product_so_far[i+1] = (i+1)*product_so_far[i]. This is induction!
In other words, we can actually identify the loop invariant of a problem by executing the proof by induction process! Not only that, the proof by induction process also identifies how to initialize the loop invariant before the loop, and how to keep the loop invariant true at the bottom of the loop!! Remember this: the Basis becomes the initialization, and the induction step becomes the increment and update step.
So, we can use Induction in loop invariant:
(1) To decompose a complex problem into the basis for induction and the induction step, which in turn allows us to define the loop invariant initialization (before the loop) and loop invariant increment and update (inside the loop).
(2) To
debug a loop to find out whether it is logically or mathematically sound—to
find out whether the goal is what we want to achieve.
(3)
To find out whether the behavior of a process is as
expected, according to what can be induced.
Note that in the real world, what can be proved through induction may
not hold true – for example, heating water.
Suppose we heat up a glass of water, whose temperature is set at 30
degrees initially. If we keep adding
heat to the water, it heats up. If we
provide constant heat energy to it, it will heat up constantly. And we can induce what the water temperature
is a minute into the future given what the current water temperature is. However, what happens when the current water
temperature is 100 degrees Celcius?
Based on past numbers, we can induce that the next temperature measurement
should read 100+. However, since water
turns into steam at 100 degrees Celcius, the temperature does not increase past
that!!! Thus, the conclusion we have
drawn using Induction does not hold true any more. This is one of the elegant applications of Induction. It allows us to predict some behavior and
test for that behavior. If that
behavior is not observed, then we know something unexpected has occurred. And then we know that our general guess has
been incorrect. That prompts
discovery!
Based on (Rosen 2003, Goodaire
and Parmenter 2002, and Ross and Wright 1988).