CS 227 Lecture -*- Outline -*-
focus: exponential complexity of computation tree for recursive version,
and designing the iteration
* Analyzing the Fibonacci Algorithm (4.6)
Italian "Son of Bonacci" = filis Bonacci. Mathematician.
-------------
FIBONACCI NUMBERS
Each pair of rabbits
has 2 kids.
Mate every month
Bear young 2 months after birth
# of pairs of rabbits:
1 ; end of month 1
1 ; end of month 2
2 = 1 + 1 ; month 3
3 = 1 + 2 ; month 4
5 = 2 + 3 ; month 5
8 = 3 + 5 ; month 6
13 = 5 + 8 ; month 7
-------------
Q: How many at the end of month 8?
-------------
Recursive Definition
(fib 0) = 0
(fib 1) = 1
(fib n) = (+ (fib (- n 1))
(fib (- n 2)))
for n > 1.
-------------
Note the need for 2 base cases.
** The program
-------------
(define fib ; TYPE: (-> (natural) natural)
(lambda (int)
(if (< int 2)
int
(+ (fib (- int 1))
(fib (- int 2))))))
-------------
try to compute on-line
(fib 5)
(fib 10)
then (fib 100)
while that's running... yawn, check your watch,
then look start to analyze it.
** Analysis
Trace how this is executed by drawing
tree for (fib 4).
- Point out it is a *binary tree*
Exercise: Draw tree for (fib 5)
using (fib 4) as subtree.
Draw tree for (fib 6).
How many procedure calls and additions?
Let (calls-fib n) be the # of procedure calls
to compute (fib n).
Can see by inspection that
-------------
HOW EXPENSIVE?
(calls-fib 0) = 1
(calls-fib 1) = 1
(calls-fib n)
= (add1 (+ (calls-fib (- n 1))
(calls-fib (- n 2))))
-------------
Similarly
-------------
(adds-fib 0) = 0
(adds-fib 1) = 0
(adds-fib n)
= (add1 (+ (adds-fib (- n 1))
(adds-fib (- n 2))))
-------------
--------------------
GRAPH OF TABLE 4.22
140 a f
130
120
110 c
100
90 a f
80
70 c
60
50 a f
40 c
30 a f
20 c a f
10 c f f
0 f f f f f f
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
n
-------------------
key: f = (fib n), c = (calls-fib n),
a = (adds-fib n)
Point out how adds-fib is just about 1 behind calls-fib
** An iterative version of fib
Need to design a sequence of numbers that will compute it.
- since need previous 2 fib numbers,
need to use *two* accumulators.
Let's play with this for a while, how to start?
pick the first 2 fib numbers
-------------
initial values
for the accumulators
acc1 acc2
0 1
= (fib 0) = (fib 1)
-------------
What could we do? Well, we get the next number by adding them
but then if we want to do this again, need to save the
previous one.
So 1 step of the iteration
- replace acc1 with old value of acc2 (remains 1 behind)
- replace acc2 with acc1 + acc2
have to do this simultaneously.
(advantage of recursion over assignment here)
Emphasize that this sequence has to be designed!!!!
how?
-------------
acc1 acc2
0 1
1 1
1 2
2 3
3 5
5 8
8 13
-------------
- can have 2 students do it (tail recursion)
-------------
(define fib2
; TYPE: (-> (natural) natural)
(lambda (n)
(if (< n 2)
n
(fib-it n 0 1))))
(define fib-it
; TYPE: (-> (natural natural natural)
; natural)
(lambda (int acc1 acc2)
(if (= int 1)
acc2
(fib-it
(sub1 int)
acc2
(+ acc1 acc2)))))
-------------
** How much more efficient is the iterative version?
Compute (fib 10) (fib 100) (fib 1000)
*** Resources used
Let (res n) be resources used to compute (fib n)
RESOURCES USED BY THE VERSIONS
ITERATIVE FULLY RECURSIVE
fib-it fib
(res n) (sub1 (add1
(* 2 n)) (* 3
(sub1
(expt
1.618
(add1 n))))
(res 10) 19 595
(res 100) 199 3.8e21
(res 1000) 1999 5.e209
(res 10000) 19999 1e2300
-------------------
RESOURCES USED BY THE VERSIONS
(res n)
1e8 r
1e7 r
1e6 r
1e5 r
1e4 r i
1e3 r i
1e2 r i
1e1 i
1 i
1 1e1 1e2 1e3 1e4
n
-------------------
key: r = fully recursive, i = iterative
if each resource unit (add or call) takes
1 nano-second (= 10^{-9}sec) to compute
-------------
TIME
if each add/call takes 1 nanosecond
ITERATIVE FULLY RECURSIVE
(time 10) 19 ns 0.6 ms
(time 100) 0.2 ms 1e7 years
(time 1000) 2 ms 1e213 years
(time 10000) 20 ms 1e2283 years
-------------
1e7 = 10^7 = 10,000,000 years
Universe is only about 5*(10^{9}) years old.
** a warning
But...
How much faster is fact using iteration?
Not much -- same number of multiplications.
Perhaps not enough to justify the example,
since recursive version of fact is so
close to definition.
A warning: it is NOT easy (or desirable)
to write iterative procedures for everything.
e.g., very hard to get deep-recursion to work iteratively
(I don't know how)
Some procedures have to go through so many gyrations to
get them to be iterative,
that they are actually slower!
(e.g. append)
** terms
-----------------------
How to talk about efficiency?
TIME
if each add/call takes 1 ns
ITERATIVE FULLY RECURSIVE
(time 10) 19 ns 0.6 ms
(time 100) 0.2 ms 1e7 years
(time 1000) 2 ms 1e213 years
(time 10000) 20 ms 1e2283 years
if each add/call takes 0.1 ns
(time 10) 1.9 ns 0.06 ms
(time 100) .02 ms 1e6 years
(time 1000) .2 ms 1e212 years
(time 10000) 2.0 ms 1e2282 years
-------------------------
So there is a qualitative, not just a quantative difference
in the efficiency; even if we get a computer that is 10 times
faster, the fully recursive version is still intolerable.
Constant factors don't matter, cause wiped out by faster machine.
Hence we summarize in big O notation
---------------
Big O Notation
ITERATIVE FULLY RECURSIVE
fib-it fib
(res n) (sub1 (add1
(* 2 n)) (* 3
(sub1
(expt
1.7
(add1 n))))
(n+1)
2n-1 3(1.7 -1)+1
= n
3(1.7(1.7) -1)+1
n
order O(n) O(1.7 )
Def: the resources (res n),
used by an input of size n,
are of order O(f(n)) iff
there is some K such that
for all sufficently large n,
(res n) <= K*f(n)
----------------
The idea is to ignore constant factors and to look at the
term containing n that makes the biggest difference.
2 2 2
100 n and 0.5 n are both O(n )
But don't ignore constant base of exponent or exponent
n**2 and n**3 are not the same, neither are 1.7**n and 2**n
n
So for the recurive fibonacci, is O(1.7 )
For the iterative, it's O(n)
n
Classes: O(n) is linear, O(c ) for c>1 is exponential
exponential algorithms are generally impractical as above.
** Using accumulators for list operations
-------------
(define reverse
; TYPE: (-> ((list T)) (list T))
(lambda (ls)
(reverse-it ls '())))
(define reverse-it
; TYPE: (-> ((list T) (list T))
; (list T))
(lambda (ls acc)
(cond
((null? ls) acc)
(else (reverse-it (cdr ls)
(cons (car ls)
acc))))))
-------------
Iterative reverse is O(n)
Recursive version is O(n*n).
** summary
Q: What is iteration good for?
Q: What is it not good for?
So useful, but not always easy or faster
The point is rather that the design of faster processes (algorithms)
makes all the difference. Iteration is just the way we did it
in this case.